Skip to content

fix(dist): duplicate group routes in one batch must not throw NPE - #298

Open
ImDanXie wants to merge 2 commits into
apache:mainfrom
ImDanXie:fix/dist-dup-group-route
Open

ImDanXie wants to merge 2 commits into
apache:mainfrom
ImDanXie:fix/dist-dup-group-route

Conversation

@ImDanXie

@ImDanXie ImDanXie commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #299

Summary

When a tenant batch contains two identical group (shared subscription) MatchRoutes, batchAddRoute throws NullPointerException and leaves the route cache inconsistent with the KV store.

Baseline: main @ eef5e3a.

Root cause

Group routes record "route to its index in the request" in a Map of MatchRoute to Integer. MatchRoute is a protobuf message and compares by value, so two identical group routes in the same batch collapse into a single entry, keeping only the later index. The earlier index remains null in the Code array:

groupMatchRecords.computeIfAbsent(new GlobalTopicFilter(tenantId, routeDetail.matcher()),
    k -> new HashMap<>()).put(route, i);   // collapses duplicates, keeps later i
...
for (BatchMatchReply.TenantBatch.Code code : codes) {
    batchBuilder.addCode(code);           // NPE on the null slot
}

Normal routes are unaffected - that branch explicitly assigns codes[i] = OK for every index in a loop. The Group branch was missed.

The proto field is documented as a deduplicated list, but the client BatchMatchCall.makeBatch does not deduplicate. The Normal branch already has explicit dedup handling and a test for it (testAddNormalRouteDuplicatedInOneBatchOnlyCountOnceAndRefresh); the Group branch was missed.

Consequence

The NPE is thrown before MutationResult is returned, but rangeWriter.done() still commits the buffered writes (normal routes + RouteGroup records land in KV), while the co-processor post-actions (routeCache.refresh, deliverExecutorGroup.refreshOrderedSharedSubRoutes, tenant counters, refreshFact) never execute. The batch is reported as ERROR to the client while the route cache remains inconsistent with the KV store until the cache expires - during which publishes are routed using stale data.

Fix

Record all positions for a route rather than only the last one - the value type of both group maps changes from Integer to List of Integer, and the consumer assigns the computed code to every recorded index. The same change applies to delGroupMatchRecords in batchRemoveRoute.

Testing

Two symmetric regression tests in DistWorkerCoProcTest:

  • testAddGroupRouteDuplicatedInOneBatchAllPositionsGetOk - add path: two identical group routes, both positions must receive OK.
  • testRemoveGroupRouteDuplicatedInOneBatchAllPositionsGetOk - unmatch path: two identical group MatchRoutes; the reader returns a RouteGroup protobuf containing the receiver, so the removal path is exercised.

Control experiment: on unfixed code both tests fail (NPE / null slot); with the fix all 19 tests in DistWorkerCoProcTest pass.

…plicate group routes

When a batch contains two identical group MatchRoutes (protobuf value
equality collapses them into one map entry, keeping only the later
index), the earlier index remains null in the Code[] array and
batchBuilder.addCode(code) throws NPE. The exception escapes before
MutationResult is returned, but rangeWriter.done() still commits the
buffered writes - leaving route cache / range fact inconsistent with
the KV store until the cache expires.

Normal routes are unaffected: that branch explicitly assigns
codes[i] = OK for every index. The Group branch was missed.

Record all positions for a route (List<Integer> instead of Integer)
and assign the computed code to every recorded index. The same change
applies to batchRemoveRoute (delGroupMatchRecords).

Regression test: testAddGroupRouteDuplicatedInOneBatchAllPositionsGetOk.
Control experiment: fails on unfixed code (NPE), passes with the fix.
Full DistWorkerCoProcTest (18) passes.
…unmatch path)

Symmetric regression test for batchRemoveRoute: two identical group
MatchRoutes in the same tenant batch. The reader returns a RouteGroup
protobuf containing the receiver so the removal path is exercised.
Without the fix, the earlier index remains null and the NPE path is
the same as in the add path. Control: both tests fail on unfixed code.
@ImDanXie
ImDanXie force-pushed the fix/dist-dup-group-route branch from 966a33d to 2a9e2e6 Compare September 24, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] DistWorker: duplicate group routes in one tenant batch cause NPE and route-cache inconsistency

1 participant