[deps] Upgrade Hudi to 0.14.1 - #878
Open
the-other-tim-brown wants to merge 1 commit into
Open
Conversation
Bump the hudi.version property from 0.14.0 to 0.14.1 on branch-0.4. 0.14.2 was evaluated first but regresses the column stats index. Its getColumnsToIndex filter resolves configured column names with Schema.getField(), which matches top-level names only, so any nested column configured via hoodie.metadata.index.column.stats.column.list is silently dropped. 0.14.1 does not contain that code. All Hudi coordinates across the reactor resolve through this single property, so no other changes are required. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CyVF3fyK2VQE7DPd31mRLv
the-other-tim-brown
force-pushed
the
hudi-0.14.2
branch
from
August 2, 2026 01:32
936bda9 to
79b7f27
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Bumps the
hudi.versionproperty onbranch-0.4from0.14.0to0.14.1.Why not 0.14.2
0.14.2 was the original target, but it regresses the column stats index and breaks
TestHudiFileStatsExtractor.columnStatsWithMetadataTable(9 indexed columns → 4).The cause is apache/hudi#18132 "fix: timestamp logical types" (merged 2026-04-10), a combined backport of #13711 and #14161. It rewrote
HoodieTableMetadataUtil.getColumnsToIndexto filter configured columns through:Schema.getField()matches top-level field names only. Any nested column configured in dot notation viahoodie.metadata.index.column.stats.column.listresolves tonulland is silently dropped from the index.TestAbstractHudiTableconfigures exactly such a list, deliberately — "all the columns in dot notation so that all columns will have stats saved to metadata table for more thorough testing."Verified against the test schema — three column-list configurations, all yielding the same result on 0.14.2:
The survivors are exactly the top-level primitives that are not timestamp-millis:
long_field,key,date_field,decimal_field. Dropped are the four nested paths plustimestamp_field. No configuration recovers them — the only code path that skips the filter is.orElse(targetColumns), reached only when the writer schema fails to resolve.Excluding timestamp-millis columns is intended and is shared with mainline. Dropping nested columns is not. Hudi master resolves the same lookup with a nested-aware helper:
So this is a defect specific to the 0.14.x backport, which substituted the top-level-only
getField. It is not tracked upstream: PR #18132 carries no JIRA key, ASF JIRA lists exactly one issue againstfixVersion = 0.14.2(HUDI-8066, an unrelated Flink cherry-pick), and the Hudi website publishes no release notes for 0.14.1 or 0.14.2. Since 0.14.x is dormant and the bug does not exist on mainline, no fix is pending.0.14.1 predates all of this and is unaffected.
Trade-off
Staying on 0.14.1 forgoes 0.14.2's timestamp logical-type correctness fixes for tables whose timestamp-millis columns were mislabeled as micros. That was judged the better trade for a maintenance branch than silently losing column stats coverage. Happy to revisit if reviewers weigh it differently.
Scope
One-line change. Every Hudi coordinate in the reactor already resolves through
${hudi.version}—xtable-core,xtable-aws,xtable-hive-metastore,xtable-hudi-support-extensions,xtable-utilities, andxtable-spark-runtimeall inherit it, so no other file needs updating.Testing
xtable-coretest suite passes locally against 0.14.1, including all 4 tests inTestHudiFileStatsExtractor, with all 9 expected columns present in the index. No test changes were needed. CI covers the full./mvnw clean installsuite plus the Spark 3.4/3.5xtable-spark-runtimebundle ITs.Intentionally not changed
features-and-limitations.md,athena.md,glue-catalog.md,spark.md,how-to.md) state Hudi 0.14.0 is required when reading a Hudi target table. That is a minimum-reader-version statement, not the build dependency, so raising it would incorrectly narrow what users can read with.demo/notebook/demo.ipynbpins Hudi 0.14.0 but is already stale (it referencesxtable-*-0.2.0-SNAPSHOT.jar). Updating only its Hudi line would be a half-fix; happy to refresh it separately.🤖 Generated with Claude Code
https://claude.ai/code/session_01CyVF3fyK2VQE7DPd31mRLv