Skip to content

Add B+Tree deletion with borrowing, merging and root collapse - #8

Merged
suhailopensource merged 2 commits into
mainfrom
step/0x08-deletion
Sep 12, 2026
Merged

suhailopensource merged 2 commits into
mainfrom
step/0x08-deletion

Conversation

@suhailopensource

@suhailopensource suhailopensource commented Sep 12, 2026 •

Copy link
Copy Markdown
Member

B+Tree deletion

The tree could grow but not shrink. This adds Delete, which is the half of a
B+Tree that most from-scratch implementations skip — and the half where the
interesting failure modes live.

Insert has one failure mode and one answer. Delete has one failure mode and
three answers, any of which can cascade to the root.

insert                     delete
does it fit? done          still enough keys? done
too full -> split          too empty -> borrow from the left sibling
                                     or borrow from the right sibling
                                     or merge with a sibling
                           and a merge may cascade upward
                           and the root may collapse

Six cases against two, and every one of the six behaves differently for a leaf
than for an internal node.

Underflow, and the three repairs

A node has underflowed when it drops below minKeys. repair tries three
things in a fixed order, cheapest first:

  1. Borrow from the left sibling if it has a key to spare.
  2. Borrow from the right sibling if it has a key to spare.
  3. Merge with a sibling — only when neither can spare one.

The order matters for cost, not correctness. Borrowing touches two nodes and
rewrites one separator. Merging touches two nodes and deletes a separator from
the parent, which is what can leave the parent underflowed, repeating the same
three-way decision one level up.

The demo reports what actually happened from the engine's own counters rather
than from a script, so it cannot narrate a repair that did not occur. At order 4
(max 3 keys, min 1), one delete sequence exercises all four outcomes:

delete 030 -> fitted, no repair needed
delete 040 -> BORROWED from the LEFT sibling
delete 010 -> MERGED two nodes into one
delete 020 -> BORROWED from the RIGHT sibling
delete 050 -> MERGED two nodes into one
delete 060 -> BORROWED from the RIGHT sibling
delete 070 -> MERGED two nodes into one, then the ROOT COLLAPSED

Root collapse is the mirror of root split, and the only way the tree gets
shorter. The tree grows from the root and it shrinks back to the root:

2000 keys      -> height 7
delete 1990    -> height 3
along the way: 1487 borrows, 1485 merges, 4 root collapses

The leaf/internal asymmetry

Deletion's asymmetry is the exact mirror of the one already in split.

leaf internal
split middle key is copied up middle key moves up
merge separator is discarded separator is pulled down between halves
borrow separator is recomputed separator rotates through the parent

The reason is the same in all six cells: a leaf holds the only copy of real
data, an internal node holds only a signpost. A leaf merge discards the
separator because both keys still exist below it. An internal merge must pull
the separator down, because up there it was the only copy.

Ghost separators

A separator only has to point a search the right way. It does not have to name a
key that still exists.

Deleting every key that appears as a separator and then re-checking:

deleted all 49 keys that appeared as separators
49 of the 49 separators left now name keys that are gone
lookups that return the wrong answer: 0 of 100
validate: <nil>

030 as a separator never promised "030 exists" — only "below 030 go left,
030 and above go right", which stays true after 030 is gone. This is why
deletion never walks back up to repair separators, and it is a large amount of
work the algorithm gets to skip. A stale separator is correct, not a bug.

Why merging is optional, and who skips it

Merging is what keeps the index dense as rows are deleted:

                     leaves   keys   fill
4000 keys            500      4000   53%
after deleting half  249      2000   54%

251 leaves handed back with the fill rate unchanged. Without merging, the leaf
count would have stayed at 500 and the fill would have halved — the index
growing while the data shrinks.

Postgres's B-Tree nevertheless does not merge partially-empty pages; it only
reclaims pages that are entirely empty. That is precisely why index bloat is an
operational concern there and why REINDEX exists. Merging costs write
amplification and page locks on a hot path, and that trade is not obviously
worth making. This implementation merges; the demo shows what that buys, and
the comparison shows what it costs.

index.Index is now satisfied

With Delete in place, btree.Index wraps the tree to satisfy the interface
declared back in 0x00, asserted at compile time with
var _ index.Index = (*Index)(nil).

The wrapper exists so the natural API can stay natural: Get returns a bool
because an in-memory miss is not exceptional, and Put cannot fail, while the
interface wants errors. The adapter translates between the two rather than
bending either.

How this is verified

  • A reference map is maintained alongside the tree through 4,000 random
    insert/delete operations at five orders
    , with Validate() after every
    operation and an exhaustive Get over the whole key space at the end.
  • TestMinKeysHoldsForEveryOrder inserts 400 and deletes 400 for every order
    from 3 to 40, checking the minimum-occupancy invariant throughout.
  • FuzzInsertDeleteAgainstReference ran 344,719 executions, each up to
    3,000 operations, validating after each one.
  • Validate() checks eight structural invariants, including that the leaf chain
    still reaches every leaf exactly once after merges splice it.

Then the test suite itself was audited by mutation testing — deliberately
breaking the implementation and checking whether the tests notice. Every package
was swept. Three mutations survived, and all three are killed by tests added
here:

survivor verdict
Next: single hop instead of a loop harmless today, but only because deletion cannot yet strand empty leaves. TestIteratorSkipsSeveralEmptyLeavesInARow splices two empty leaves into the chain by hand; the single-hop version panics.
Validate: drop the subtree bounds check a genuine gap. No test had ever produced a tree that violated its subtree bounds, so the most important invariant in the file had never once fired. TestValidateCatchesABadSeparator corrupts a separator in both directions.
NewLayout: accept a negative record size equivalent — a second guard catches it identically. The guard still earns its place because the fallback error message is nonsense (-5-byte record needs a -5-byte slot), so the test now asserts the message names the record size.

That second one is worth stating plainly: a split or borrow that promotes the
wrong key leaves every node locally sorted while silently routing searches
to the wrong leaf. Ordering, depth, and the leaf chain would all still pass.

Not in this change

No persistence — this tree is still pointers in memory, and nodes become pages
in a later step. No concurrency or locking. No rebalancing policy beyond the
fixed borrow-then-merge order.

Verification

go build ./...   ok
go vet ./...     clean
gofmt -l .       0 files
go test ./...    ok, all 5 packages
go test -race    clean

99 tests, 9 fuzz targets, 11 benchmarks. Run the walkthrough with
make demo-delete.

The tree could grow but not shrink. Insert has one failure mode and one
answer; delete has one failure mode and three, any of which can cascade
to the root. Six cases against two, and each behaves differently for a
leaf than for an internal node.

repair tries the cheapest fix first: borrow from the left sibling, else
borrow from the right, else merge. Borrowing rewrites one separator.
Merging deletes one, which is what can leave the parent underflowed and
repeat the same decision one level up. When the root is left with no
keys it collapses into its only child, which is the mirror of a root
split and the only way the tree gets shorter. 2000 keys at height 7,
after deleting 1990 height 3, via 1487 borrows, 1485 merges and 4
collapses.

The leaf/internal asymmetry mirrors split exactly. A leaf merge discards
the separator because both keys still exist below it; an internal merge
pulls it down between the halves, because up there it was the only copy.

A separator does not have to name a key that exists. Deleting all 49
keys that appeared as separators leaves 49 stale separators, 0 wrong
answers out of 100 lookups, and Validate clean. 030 never promised "030
exists", only "below 030 go left", which survives the delete. That is
why deletion never walks back up to repair separators.

Merging keeps the index dense: 4000 keys over 500 leaves at 53% fill
becomes 2000 keys over 249 leaves at 54%, handing back 251 leaves.
Postgres's B-Tree does not do this, reclaiming only entirely empty
pages, which is why index bloat is operational there and REINDEX exists.
Merging costs write amplification and page locks on a hot path.

With Delete in place btree.Index satisfies the index.Index interface
declared in 0x00, asserted at compile time. The wrapper exists so the
natural API can stay natural: Get returns a bool, Put cannot fail.

Verified against a reference map through 4,000 random operations at five
orders with Validate after every one, an exhaustive Get at the end, and
FuzzInsertDeleteAgainstReference at 344,719 executions of up to 3,000
operations each.

The test suite itself was then audited by mutation testing across every
package. Three mutations survived. Iterator.Next surviving a single-hop
rewrite was harmless only because deletion cannot yet strand empty
leaves, now covered by splicing empty leaves in by hand. Validate
surviving the removal of its subtree bounds check was a real gap: no
test had ever produced a tree that violated those bounds, so the most
important invariant in the file had never once fired. A split that
promotes the wrong key leaves every node locally sorted while routing
searches to the wrong leaf, and ordering, depth and the chain all still
pass. The third, in slots, is an equivalent mutant kept honest by
asserting the error message.

99 tests, 9 fuzz targets, race clean.
The B+Tree now searches, inserts, splits, scans ranges and deletes, which is the whole of Phase A. Adds a closing summary of what the six steps add up to, and records the gap that opens Phase B: every node is still a Go pointer, so nothing survives a restart.
@suhailopensource
suhailopensource merged commit e597aa0 into main Sep 12, 2026
1 check passed
@suhailopensource
suhailopensource deleted the step/0x08-deletion branch September 12, 2026 05:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant