Skip to content

fix(firestore-bigquery-change-tracker): partitioning e2e suite fails on next and the package never runs in CI #3136

Description

@cabljac

Running the change tracker's jest suite on next (c3522bf) against live BigQuery fails 8 of the 25 tests in src/__tests__/bigquery/e2e.test.ts, all in the partitioning block. Nobody noticed because the root jest.config.js projects list includes */functions and gen-schema-view but not firestore-bigquery-export/firestore-bigquery-change-tracker, so CI has never run this package's tests. The 12 failures in backupSettings.test.ts that #3134 had to fix went through CI green for the same reason.

The 8 failures have four causes, all introduced by the partitioning refactor 442722a (#2447) and one follow-on in 11b3406:

  1. Package regression. src/bigquery/partitioning/config.ts determineType only rejects a falsy or NONE granularity, so an unrecognised value such as UNKNOWN is classed as ingestion time and src/bigquery/partitioning/index.ts sends timePartitioning: { type: "UNKNOWN" } to createTable, which BigQuery rejects with Invalid value for timePartitioning.type: UNKNOWN is not a valid value. The pre-refactor code whitelisted HOUR, DAY, MONTH and YEAR. Test: "does not partition with an unrecognized timePartitioning option".
  2. Four tests had their invalid inputs replaced with valid ones during the refactor but kept expect(metadata.timePartitioning).toBeUndefined(): the "without a valid timePartitioningField", "without a valid timePartitioningFieldType", "unknown timePartitioningFieldType" and "unknown timePartitioningFirestoreField" tests.
  3. Two stale assertions: the "timestamp as field name" test asserts DAY after its config moved to HOUR, and "valid schema with timePartitioning only" asserts a warn log the refactor deleted; the logger spy is also never cleared between tests.
  4. 11b3406 made tableRequiresUpdate add a missing custom partition column to an existing table, so "does not update add a custom partitioning column" now sees one column added where it expects none. Intent needs deciding.

Suggested fix: validate the granularity in determineType and omit timePartitioning on an unrecognised value (the existing TODO there), restore genuinely invalid inputs in the four tests, correct the two stale assertions, clear the logger spy in beforeEach, decide test 4 against 11b3406, and add the package to the root jest projects so CI runs it. Two related gaps for the CI entry: the suite needs live BigQuery credentials, and on firebase-admin 14 the ts-jest config cannot load the ESM-only jose that firebase-functions v2 pulls in, so transformIgnorePatterns needs adjusting before the admin 14 path is covered.

Found while validating #3134; the same 8 fail with the pre-#3134 test file, so this is independent of that change.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions