Repository navigation
Allow equality-delete upserts after partition spec evolution - #4
Merged
nicosuave merged 5 commits intoOct 12, 2026
Merged
Conversation
Partition-scoped equality deletes only apply to data files written under the same spec, so upserts were rejected whenever a table had more than one spec. That blocked tables permanently even after every data file had been rewritten into the current spec. Allow the upsert when no live data files on the target branch remain under another spec, re-check on commit retry, and release 0.12.0.post2.
The pinned ref fac544c (v8.2.0) was dropped from the ASF allowlist when it expired on 2026-09-27, so asf-allowlist-check fails on every pull request. v10.2.0 is the only setup-uv entry left without an expires_at. (cherry picked from commit 39440e5)
* Infra: Replace MinIO with RustFS in the dev and integration stacks MinIO Community Edition stopped publishing images in October 2025 and the repository was archived in April 2026, so `minio/minio:latest` resolves to a frozen build that will not receive further security fixes (apache#2664). I'm choosing RustFS since that's what Polaris uses. * Infra: Rename the object store service to `object-store` Feedback from @kevinjqliu (cherry picked from commit d5474fd)
google.api_core changes the wording of its Python version FutureWarning by
date. Python 3.10 moved past the deprecation wording ("which Google will stop
supporting") into the end-of-life and non-supported wordings on 2026-10-11
and 2026-10-12, so the existing filter stopped matching and pytest turned the
warning into a collection error on 3.10. Match all three wordings.
The apache/hive:4.0.0 image is based on Debian 11, whose security repository moved to archive.debian.org. apt-get update still succeeds, but installing curl now fails with 404s from deb.debian.org/debian-security, which breaks the integration-test image build. Download the two JARs with ADD so the image no longer needs packages from Debian.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Partition-scoped equality deletes only apply to data files written under the same spec, so upserts were rejected whenever a table had more than one spec. That blocked a table permanently, even after every data file was rewritten into the current spec.
The upsert is now allowed when no live data files on the target branch remain under another spec. The check reads only manifest metadata, honors
branch, and runs again on commit retry, so a concurrent spec change plus rewrite cannot leave deletes that miss data. It raisesValidationExceptionand lists the remaining spec ids otherwise.Bumps the distribution to 0.12.0.post2 so dlt-iceberg can pin it. Sidequery needs this to move
posthog.eventsfrom daily to monthly partitions.