Skip to content

fix(bulk_load): keep all leaves on the same depth - #239

Open
jaideeppyne wants to merge 1 commit into
georust:masterfrom
jaideeppyne:fix/bulk-load-uniform-leaf-depth
Open

fix(bulk_load): keep all leaves on the same depth#239
jaideeppyne wants to merge 1 commit into
georust:masterfrom
jaideeppyne:fix/bulk-load-uniform-leaf-depth

Conversation

@jaideeppyne

Copy link
Copy Markdown

RTree::bulk_load built the tree by recursively partitioning elements and turning any cluster with ≤ MAX_SIZE elements into a leaf. When the element count made sibling clusters straddle MAX_SIZE (some just below, some just above), the larger clusters were split one level deeper than the smaller ones, so leaves ended up on different depths.

Such a tree violates the R-tree invariant that all leaves share the same depth (the invariant ParentNode::sanity_check asserts). Queries still worked, but a later insert descends assuming uniform depth and panics with "This is a bug in rstar." on reaching a node that mixes leaf and parent children. Reproducible with default parameters in 2-D, e.g. bulk-loading 25 points and then inserting two more:

let mut t = RTree::bulk_load(vec![[12,-8],[-8,-9], /* … 25 points … */]);
t.insert([-8,-1]);
t.insert([3,9]); // panicked

Fix: compute the target tree depth up front and thread it through the recursion so every branch stops on the same level. Adds a regression test and a CHANGELOG entry. Full suite + fmt + clippy pass.

This is distinct from #197 (which is about MAX_SIZE compliance in higher dimensions — the repro here keeps every child count within MAX_SIZE) and from the long-closed #45 (a reinsertion bug; here the tree is already malformed straight out of bulk_load, before any insert). It does not attempt to fully solve #197.

  • I agree to follow the project's Code of Conduct.

bulk_load built the tree by recursively partitioning the elements and
turning any cluster with at most MAX_SIZE elements into a leaf node. When
the element count made sibling clusters straddle MAX_SIZE (some just
below, some just above), the larger clusters were split one level deeper
than the smaller ones, so leaves ended up on different levels.

Such a tree violates the R-tree invariant that all leaves share the same
depth. Queries still worked, but a subsequent insert descended the tree
assuming a uniform depth and panicked with "This is a bug in rstar."
once it reached a node mixing leaf and parent children. This is
reproducible with the default parameters in two dimensions, e.g.
bulk-loading 25 points and then inserting two more.

Compute the target tree depth up front and thread it through the
recursion so every branch stops at the same level, placing all leaves on
one depth. Add a regression test.
Comment thread rstar/src/algorithm/bulk_load/bulk_load_sequential.rs
fallenmi

This comment was marked as low quality.

@jaideeppyne

Copy link
Copy Markdown
Author

Hold off on merging this for a moment, I found a regression in it.

The depth calculation I added divides the cluster size by the combined fan-out and rounds down, but ClusterGroupIterator cuts a slab of len.div_ceil(number_of_clusters_on_axis) once per axis. So it understates the largest cluster on the next level, stops one level early, and leaves nodes above MAX_SIZE.

It shows up in 2D, where master is clean. Smallest case is 25 collinear points:

let pts: Vec<[f64; 2]> = (0..25).map(|i| [i as f64, i as f64]).collect();
let tree = RTree::<[f64; 2], DefaultParams>::bulk_load_with_params(pts);
// a node ends up with 7 children, MAX_SIZE is 6

Sweeping n = 1..300 on collinear input, master has no violations and this branch has them from n = 25. My own test here doesn't catch it because it calls sanity_check::<DefaultParams>(false), and false is precisely the flag that skips the MAX_SIZE assertion. With true it panics at n = 25.

Rounding the depth up per axis instead is not enough on its own: it pushes some trees a level deeper and then nodes fall below MIN_SIZE, which is what the rounding-down comment in the patch was guarding against.

@adamreichold you suggested changing ClusterGroupIterator rather than computing the cluster count twice, and I think you were right, just for a bigger reason than the redundancy. Fixing it in the iterator is what actually resolves this.

What does work is fixing the fan-out itself. Splitting every axis by the same number gives a node clusters ^ DIMENSIONS children, which is already over MAX_SIZE as soon as there is more than one axis to split, so 3D has 8-child nodes and 6D has 64-child nodes on master today. That is #197. If instead the cuts are handed out one axis at a time, always to the axis cut least so far, and only while the product stays inside MAX_SIZE, then the largest node is 6 in 2D through 6D and leaf depth stays uniform. Full suite passes, so MIN_SIZE holds too.

I have that on a branch. Happy to either fold it into this PR, which would mean you re-reviewing since it touches the code this PR adds, or land this one first with just the depth rounding sorted and do the fan-out separately as a fix for #197. Your call on which is less annoying to review.

One thing worth flagging if we go the fan-out route: it makes 4D and up substantially faster (6D nearest-neighbour roughly halves) but 3D range queries get slower, because cutting all three axes in two needs a fan-out of 8 and MAX_SIZE is 6, so 3D loses a split that the oversized nodes were getting for free. That also answers @michaelkirk's question in #197 about splitting every axis: it is not compatible with MAX_SIZE < 2^DIMENSIONS.

I used an LLM to build the differential harness and the sweeps behind this, as with the original PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unreachable! panic when inserting points

3 participants