Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions kits/delete-user-data/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ service account for this codebase — it conflicts with that automatic setup.
| `roles/pubsub.admin` | publish/subscribe discovery and deletion topics |
| `roles/eventarc.eventReceiver` | receive Gen2 event triggers |
| `roles/run.invoker` | allow Eventarc to invoke the Gen2 Cloud Run service |
| `roles/eventarc.publisher` | publish the kit's custom Eventarc events (the Extensions platform granted this implicitly) |

## Usage

Expand Down
5 changes: 5 additions & 0 deletions kits/delete-user-data/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,11 @@ const REQUIRED_ROLES: ReadonlyArray<Role> = [
// Gen2 event triggers need Eventarc receive and run.invoker on the function SA.
"roles/eventarc.eventReceiver",
"roles/run.invoker",
// The Extensions platform granted publish rights on the extension's Eventarc
// channel implicitly from `events:` in extension.yaml. Kits get no implicit
// grant, so without this the `channel.publish()` calls in ./events fail with
// PERMISSION_DENIED and no custom event is ever delivered.
"roles/eventarc.publisher",
];

for (const role of REQUIRED_ROLES) {
Expand Down
1 change: 1 addition & 0 deletions kits/firestore-bigquery-export/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ conflicts with that automatic setup.
| `roles/datastore.user` | write failed-row records back to Firestore (only if you configure a backup collection) |
| `roles/eventarc.eventReceiver` | receive Gen2 Firestore trigger events |
| `roles/run.invoker` | allow Eventarc to invoke the Gen2 Cloud Run service |
| `roles/eventarc.publisher` | publish the kit's custom Eventarc events (the Extensions platform granted this implicitly) |
| `bigquery.googleapis.com` | mirror Firestore collection changes in BigQuery |

If the dataset lives in a different project (`BIGQUERY_PROJECT_ID`), grant the
Expand Down
5 changes: 5 additions & 0 deletions kits/firestore-bigquery-export/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,11 @@ const REQUIRED_ROLES: ReadonlyArray<Role> = [
// Gen2 Firestore triggers need Eventarc receive and run.invoker on the function SA.
"roles/eventarc.eventReceiver",
"roles/run.invoker",
// The Extensions platform granted publish rights on the extension's Eventarc
// channel implicitly from `events:` in extension.yaml. Kits get no implicit
// grant, so without this the `channel.publish()` calls in ./events fail with
// PERMISSION_DENIED and no custom event is ever delivered.
"roles/eventarc.publisher",
];
const REQUIRED_APIS = [
{
Expand Down
18 changes: 6 additions & 12 deletions kits/firestore-counter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ service account for this codebase — it conflicts with that automatic setup.
| `roles/cloudscheduler.admin` | schedule the controller that flushes shards |
| `roles/eventarc.eventReceiver` | receive Gen2 Firestore trigger events |
| `roles/run.invoker` | allow Eventarc/Scheduler to invoke the Gen2 Cloud Run service |
| `roles/eventarc.publisher` | publish the kit's custom Eventarc events (the Extensions platform granted this implicitly) |

## Usage

Expand Down Expand Up @@ -161,16 +162,6 @@ for them, so no events are published until you create a channel and put both
values in your `.env`. If you set `EVENTARC_CHANNEL` and leave
`EXT_SELECTED_EVENTS` unset, every event type is published.

### Event payloads have a different shape

The event types are unchanged, but what they carry is not. `onStart` used to
carry `{change, context}` and now carries `{data, params}`: the write is under
`data` instead of `change`, and the 1st gen `context` is gone. `onCompletion`
used to carry `{context}` and now carries `{params}` only. Anything reading
`context.eventId`, `context.timestamp`, `context.eventType` or
`context.resource` from these events needs updating; the trigger wildcards
(`collection`, `counter`, `shardId`) survive as `params`.

### Your codebase's global options apply to these functions

The functions are exported from your own functions codebase, so a
Expand Down Expand Up @@ -201,8 +192,11 @@ carry on getting them from the extension repo.
- The aggregation behaviour: inline aggregation up to 200 shards, workers above
that, 45 second self-scheduling worker runs, partial shard cleanup, and
deletion of shards once they are summed into the counter field.
- The three functions and the event types they publish, aside from the worker
and payload points above.
- The three functions, the event types they publish and their payloads:
`onStart` still carries `{change, context}` and `onCompletion` still carries
`{context}`, with `context.eventId`, `context.timestamp`, `context.eventType`,
`context.resource` and the trigger wildcards under `context.params`. Aside
from the worker point above.
Comment thread
CorieW marked this conversation as resolved.

## API surface

Expand Down
64 changes: 64 additions & 0 deletions kits/firestore-counter/src/event-context.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
/**
* Copyright 2026 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import type { FirestoreEvent } from "firebase-functions/v2/firestore";

/**
* The 1st gen `EventContext` shape that the extension published inside its
* `onStart` and `onCompletion` payloads.
*/
export interface EventContext {
eventId: string;
timestamp: string;
eventType: string;
resource: {
service: string;
name: string;
};
params: Record<string, string>;
}

/**
* Every 1st gen Firestore `onWrite` trigger reported this event type, so
* subscribers matching on `context.eventType` keep matching it.
*/
const FIRESTORE_WRITE_EVENT_TYPE = "google.firestore.document.write";

const FIRESTORE_SERVICE = "firestore.googleapis.com";

/**
* Rebuilds the 1st gen `EventContext` from a 2nd gen `FirestoreEvent`.
*
* The extension handed the 1st gen handler's `context` straight to Eventarc, so
* subscribers read `eventId`, `timestamp`, `eventType`, `resource` and `params`
* off it. `FirestoreEvent` carries the same information under different names,
* so the published payload keeps its original shape instead of following the
* 2nd gen handler signature.
*/
export function toEventContext(
Comment thread
cabljac marked this conversation as resolved.
event: FirestoreEvent<unknown, Record<string, string>>
): EventContext {
return {
eventId: event.id,
timestamp: event.time,
eventType: FIRESTORE_WRITE_EVENT_TYPE,
resource: {
service: FIRESTORE_SERVICE,
name: `projects/${event.project}/databases/${event.database}/documents/${event.document}`,
},
params: event.params,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the extension published any params here, so "still ... under context.params" in the README isn't parity, it's a superset.

In v1, context.params isn't supplied by the platform. The SDK computes it from the code-side trigger path (_makeParams, firebase-functions/lib/v1/cloud-functions.js:150). The extensions registered document(process.env.INTERNAL_STATE_PATH) (firestore-counter/functions/src/index.ts:80) and document(process.env.COLLECTION_PATH) (firestore-translate-text/functions/src/index.ts:47). Neither has a {wildcard} segment, so WILDCARD_REGEX matches nothing and params comes out {}. The yaml wildcards registered the trigger, they never reached the SDK.

Worth verifying yourself before you change anything, since you already have the deploy set up. Harmless either way, a superset breaks nobody, but the README and the description both claim a parity I don't think holds.

The more general point: the deploy test ran the kit's own toEventContext over 2nd gen events, so it can only confirm the kit is self-consistent. Nothing in it observes a real 1st gen context. If you can get one out of the emulator against the old extension, commit it as a fixture and assert event-context.test.ts against it. Then the parity claim is pinned rather than argued.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One precision after reading _makeParams properly: the only way the extension's params was non-empty is if a user put their own {wildcard} segments inside COLLECTION_PATH itself (the validation regex allows braces, and the code-side path is that env var verbatim). Even then messageId never appears, because it only exists in the yaml resource. So the exact statement is "empty by default, and never the yaml wildcards". The rest of the comment stands.

};
}
6 changes: 4 additions & 2 deletions kits/firestore-counter/src/handlers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import type { DocumentSnapshot, Firestore } from "firebase-admin/firestore";
import type { Change, FirestoreEvent } from "firebase-functions/v2/firestore";
import type { ScheduledEvent } from "firebase-functions/v2/scheduler";
import { ControllerStatus, ShardedCounterController } from "./controller";
import { toEventContext } from "./event-context";
import * as events from "./events";
import type { ResolvedCounterConfig } from "./export-config";
import { ShardedCounterWorker } from "./worker";
Expand Down Expand Up @@ -62,7 +63,8 @@ export async function handleShardWrite(
event: CounterWriteEvent,
ctx: HandlerContext
): Promise<void> {
await events.recordStartEvent({ data: event.data, params: event.params });
const context = toEventContext(event);
await events.recordStartEvent({ change: event.data, context });
const metadocRef = ctx.firestore.doc(ctx.config.internalStatePath);
const controller = new ShardedCounterController(
metadocRef,
Expand All @@ -73,7 +75,7 @@ export async function handleShardWrite(
INLINE_AGGREGATION_LIMIT,
INLINE_AGGREGATION_TIMEOUT_MS
);
await events.recordCompletionEvent({ params: event.params });
await events.recordCompletionEvent({ context });
}

export async function handleWorker(event: CounterWriteEvent): Promise<void> {
Expand Down
5 changes: 5 additions & 0 deletions kits/firestore-counter/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@ const REQUIRED_ROLES: ReadonlyArray<Role> = [
// Gen2 Firestore triggers need Eventarc receive and run.invoker on the function SA.
"roles/eventarc.eventReceiver",
"roles/run.invoker",
// The Extensions platform granted publish rights on the extension's Eventarc
// channel implicitly from `events:` in extension.yaml. Kits get no implicit
// grant, so without this the `channel.publish()` calls in ./events fail with
// PERMISSION_DENIED and no custom event is ever delivered.
"roles/eventarc.publisher",
];

for (const role of REQUIRED_ROLES) {
Expand Down
60 changes: 60 additions & 0 deletions kits/firestore-counter/tests/event-context.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
/**
* Copyright 2026 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import { describe, expect, test } from "vitest";
import { toEventContext } from "../src/event-context";

/**
* The extension published its 1st gen handler's `context` verbatim, so these
* assertions pin the fields subscribers read off it.
*/
describe("toEventContext", () => {
const event = {
id: "event-1",
time: "2026-01-01T00:00:00.000Z",
project: "demo-project",
database: "(default)",
document: "pages/home/_counter_shards_/0000",
params: { collection: "pages", counter: "home", shardId: "0000" },
} as any;

test("rebuilds the 1st gen event context from a 2nd gen event", () => {
expect(toEventContext(event)).toEqual({
eventId: "event-1",
timestamp: "2026-01-01T00:00:00.000Z",
eventType: "google.firestore.document.write",
resource: {
service: "firestore.googleapis.com",
name: "projects/demo-project/databases/(default)/documents/pages/home/_counter_shards_/0000",
},
params: { collection: "pages", counter: "home", shardId: "0000" },
});
});

test("names the resource under the event's own database", () => {
const context = toEventContext({ ...event, database: "counters" });

expect(context.resource.name).toBe(
"projects/demo-project/databases/counters/documents/pages/home/_counter_shards_/0000"
);
});

test("passes the trigger wildcards through unchanged", () => {
const params = { collection: "docs", counter: "a/b/c", shardId: "0001" };

expect(toEventContext({ ...event, params }).params).toEqual(params);
});
});
53 changes: 49 additions & 4 deletions kits/firestore-counter/tests/events.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,17 @@
*/

import { afterEach, beforeEach, describe, expect, test, vi } from "vitest";
import { toEventContext } from "../src/event-context";

/** A shard write on the `{collection}/{counter=**}/_counter_shards_/{shardId}` trigger. */
const SHARD_WRITE = {
id: "event-1",
time: "2026-01-01T00:00:00.000Z",
project: "demo-project",
database: "(default)",
document: "pages/home/_counter_shards_/0000",
params: { collection: "pages", counter: "home", shardId: "0000" },
} as any;

const publish = vi.fn();
const channel = vi.fn(() => ({ publish }));
Expand Down Expand Up @@ -81,11 +92,16 @@ describe("event publishing", () => {
test("publishes start events", async () => {
const events = await setupEnabledEvents();

await events.recordStartEvent({ params: { shardId: "0000" } });
const context = toEventContext(SHARD_WRITE);

await events.recordStartEvent({
change: { before: {}, after: {} },
context,
});

expect(publish).toHaveBeenCalledWith({
type: "firebase.extensions.firestore-counter.v1.onStart",
data: { params: { shardId: "0000" } },
data: { change: { before: {}, after: {} }, context },
});
});

Expand Down Expand Up @@ -119,12 +135,41 @@ describe("event publishing", () => {
test("publishes completion events", async () => {
const events = await setupEnabledEvents();

await events.recordCompletionEvent({ params: { shardId: "0000" } });
const context = toEventContext(SHARD_WRITE);

await events.recordCompletionEvent({ context });

expect(publish).toHaveBeenCalledWith({
type: "firebase.extensions.firestore-counter.v1.onCompletion",
data: { params: { shardId: "0000" } },
data: { context },
});
});

test("puts the whole 1st gen context on the wire", async () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few small ones, none blocking:

  • This test mocks firebase-admin/eventarc, so the real serializer (toCloudEventProtoFormat) never runs, and recordStartEvent is a pass-through. It ends up being the event-context.test.ts assertion again with a hand-rolled JSON.stringify. Not wrong, but the comment claims more than it proves. Either drop it, or publish something where JSON semantics actually differ from toEqual (an error-bearing payload would).
  • No failure-mode coverage. strict is off, so timestamp: undefined compiles fine and the key just vanishes from the wire. One test for a missing event.time would catch that.
  • SHARD_WRITE here (:21-28) and the fixture in event-context.test.ts:25-32 are both as any. Translate centralises this in makeEvent (tests/helpers.ts:102), still a cast but only in one place. Worth giving counter the same.
  • expectedEventContext (tests/helpers.ts:122) rebuilds the projects/.../databases/.../documents/... template at :131 from the same parts the source uses, so the same mistake in both places passes. A literal expected string is stronger.
  • firestore-translate-text/README.md:216 says "The four event payloads:" and then describes two.

const events = await setupEnabledEvents();
const context = toEventContext(SHARD_WRITE);

await events.recordStartEvent({
change: { before: {}, after: {} },
context,
});
await events.recordCompletionEvent({ context });

// `firebase-admin` sends the payload as `JSON.stringify(data)`, so this is
// what a subscriber of the extension's events actually reads.
expect(publish.mock.calls.length).toBe(2);
for (const [event] of publish.mock.calls) {
expect(JSON.parse(JSON.stringify(event.data)).context).toEqual({
eventId: "event-1",
timestamp: "2026-01-01T00:00:00.000Z",
eventType: "google.firestore.document.write",
resource: {
service: "firestore.googleapis.com",
name: "projects/demo-project/databases/(default)/documents/pages/home/_counter_shards_/0000",
},
params: { collection: "pages", counter: "home", shardId: "0000" },
});
}
});

test("does nothing before the channel is set up", async () => {
Expand Down
28 changes: 22 additions & 6 deletions kits/firestore-counter/tests/handlers.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -113,24 +113,40 @@ describe("handleSchedule", () => {
describe("handleShardWrite", () => {
test("aggregates continuously and records lifecycle events", async () => {
const event = {
id: "event-1",
time: "2026-01-01T00:00:00.000Z",
project: "demo-project",
database: "(default)",
document: "pages/home/_counter_shards_/0000",
data: { after: { exists: true } },
params: { shardId: "0000" },
params: { collection: "pages", counter: "home", shardId: "0000" },
} as any;

await handleShardWrite(event, makeCtx());

// The extension published the 1st gen `{change, context}` payload, so the
// kit rebuilds the same shape rather than exposing the 2nd gen event.
const context = {
eventId: "event-1",
timestamp: "2026-01-01T00:00:00.000Z",
eventType: "google.firestore.document.write",
resource: {
service: "firestore.googleapis.com",
name: "projects/demo-project/databases/(default)/documents/pages/home/_counter_shards_/0000",
},
params: { collection: "pages", counter: "home", shardId: "0000" },
};

expect(events.recordStartEvent).toHaveBeenCalledWith({
data: event.data,
params: event.params,
change: event.data,
context,
});
expect(aggregateContinuously).toHaveBeenCalledWith(
{ start: "", end: "" },
200,
60000
);
expect(events.recordCompletionEvent).toHaveBeenCalledWith({
params: event.params,
});
expect(events.recordCompletionEvent).toHaveBeenCalledWith({ context });
});
});

Expand Down
1 change: 1 addition & 0 deletions kits/firestore-send-email/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ service account for this codebase — it conflicts with that automatic setup.
| `roles/datastore.user` | read mail documents and write delivery status |
| `roles/eventarc.eventReceiver` | receive Gen2 Firestore trigger events |
| `roles/run.invoker` | allow Eventarc to invoke the Gen2 Cloud Run service |
| `roles/eventarc.publisher` | publish the kit's custom Eventarc events (the Extensions platform granted this implicitly) |

## Usage

Expand Down
5 changes: 5 additions & 0 deletions kits/firestore-send-email/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,11 @@ const REQUIRED_ROLES: ReadonlyArray<Role> = [
// Gen2 Firestore triggers need Eventarc receive and run.invoker on the function SA.
"roles/eventarc.eventReceiver",
"roles/run.invoker",
// The Extensions platform granted publish rights on the extension's Eventarc
// channel implicitly from `events:` in extension.yaml. Kits get no implicit
// grant, so without this the `channel.publish()` calls in ./events fail with
// PERMISSION_DENIED and no custom event is ever delivered.
"roles/eventarc.publisher",
];

for (const role of REQUIRED_ROLES) {
Expand Down
Loading