Skip to content

A spanning import can strand a segment in the active tier that no key filter covers; its in-window keys read as absent #90

Description

@PaulSnow

Found by review, not by failure. Latent today, and masked by a second defect.

The stranding

handoffBelowWindow (segstore.go:1137) stops at the first segment whose oldest block is inside the window:

for _, seg := range s.active {
    if seg.meta.first() >= start { break }  // "Ordered: the rest are inside the window too"
    n++
}

That assumes s.active is ordered by first(). Seals satisfy it: a seal has Span == 0, so first() == Height. An import does not. ImportSegmentFile appends a segment whose Height is the highest in the store (it must be) but whose Span > 0 puts its first() below segments already in active.

Concretely, with N=20: active holds P{Height:125, Span:0, first:125}. Import Q{Height:130, Span:28, first:102}; at tierStart=100 it joins active correctly. The block reaches 140, wanted=[120,140], the filter at 100 that held Q's keys is dropped, the new filter at 120 is built only from segments with first() >= 120 so Q is excluded, and the handoff loop breaks on P.first()=125 >= 120 before ever reaching Q.

Q is now in active, below the window, covered by no filter. findActive (segstore.go:1901) returns immediately when the filters say no, without walking active at all, so Q's keys read as absent — and they are keys at blocks 120..130, inside the window, so this is a false absent by the filters' own claim rather than a windowing decision. Put of one of those keys with a different value is then accepted, which is the immutability breach the check exists to prevent.

It self-heals on reopen, because load re-tiers by first().

Why it is latent

SegmentInfo (segment.go:45) has no Span field, so a block export drops it and every imported segment arrives claiming one block. That keeps active ordered — and is itself wrong, because spec 1.4 gives Span exactly so a filter cannot claim a merged segment whose older keys it never saw. The two defects mask each other, and closing either one alone exposes the other.

The same assumption again

SetFilterBlocks (keyfilter.go:330) scans s.history backwards for first() >= start and stops at the first that fails. s.history is ordered by (Height, Seq), not by first(), so a qualifying segment before that point stays in history, uncovered by the rebuilt filters. Its own doc comment says this is the thing it is protecting against.

Notes

No test imports a spanning segment; keyfilter_test.go:550 and segstore_test.go:268 import Span == 0 seals only. A fix wants a test that imports Span > 0 and then rolls the window past it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions