NanoVDB: handle empty partitions in the distributed merge path search

[harrism]

What

tools::cuda::DistributedPointsToGrid's merge path search (mergePath) mishandles an empty partition. Its bounds checks compare against keys1Count - 1 / keys2Count - 1 on size_t counts, so a zero count underflows to SIZE_MAX and the guards never fire; the binary search then reads the empty array (and out of bounds past either array) and returns wrong or garbage intervals — an inverted split in one direction, underflow garbage in the other.

An empty partition is reachable from radixSortAsync's ceil-split whenever a device receives no items — e.g. sorting a single element across two devices gives counts [1, 0].

The fix returns the trivial split before the search: with an empty side, every element up to the diagonal comes from the non-empty side.

Testing

New test TestNanoVDBMultiGPU.MergePathEmptyPartition — runs on a single GPU (mergePathKernel is a plain kernel over pointers), covering empty-right, empty-left, both-empty, and a non-empty control at the median diagonal. The adjacent memory is filled with controlled sentinels so the out-of-bounds search fails deterministically rather than by luck:

• Without the fix: empty-right returns the inverted split (0, 2) instead of (2, 0); empty-left returns underflow garbage in both intervals.
• With the fix: all cases pass, and compute-sanitizer --tool memcheck reports 0 errors.
• Full nanovdb_test_mgpu suite: 8/8 pass (single GPU; RTX 6000 Ada, CUDA 12.6).

Found while investigating the (environmental, since closed) multi-GPU failures in #2245 — this defect is independent of that machine issue and affects any multi-GPU sort with an empty partition.

Non-CUDA builds

No impact on NANOVDB_USE_CUDA=OFF — the change is confined to a CUDA-only header and the multi-GPU test.

🤖 Generated with Claude Code

[matthewdcong]

Do we need to launch the kernel if either keys1Count or keys2Count is 0? I think this condition could be moved into the host code prior to kernel launch.

[harrism]

Good call — done in 3a8f760. The launch site now skips the kernel when either count is zero and computes the trivial split on the host (mergePathTrivial, kept as a small named function so the unit test can exercise it directly); mergePath reverts to the unguarded search with the non-empty precondition documented. The test's empty cases are now pure host calls, with the device search kept as the non-empty control. Verified: nanovdb_test_mgpu 8/8 (single GPU), compute-sanitizer clean.

[harrism]

Indeed. Fixed in 3a8f760

[harrism]

@kmuseth friendly ping — open since July 7, no review yet.

This one is a latent out-of-bounds read rather than a feature: mergePath compares against keys2Count - 1 on a size_t, so a zero count underflows to SIZE_MAX, the guards never fire, and the binary search reads past the end of the array and returns garbage intervals.

An empty partition is reachable in practice — radixSortAsync's ceil-split hands one device nothing whenever the item count is small relative to the device count (sorting a single element across two GPUs gives [1, 0]).

Worth noting for scheduling: the new test MergePathEmptyPartition runs on a single GPU, so it isn't blocked on multi-GPU hardware. It's +67/-1 across 2 files.