Skip to content

fix(grpc): end streams with NOT_FOUND at unindexed heights - #65

Open
setavenger wants to merge 1 commit into
integrationfrom
fix/streams-not-found-unindexed
Open

setavenger wants to merge 1 commit into
integrationfrom
fix/streams-not-found-unindexed

Conversation

@setavenger

Copy link
Copy Markdown
Owner

Warning

Deploy ordering: do not deploy this to users before the blindbit-rs client fix (setavenger/blindbit-rs#20) is released. Current Rust clients call .unwrap() on every stream message. With this change they get a NOT_FOUND status instead of an OK message, so they would panic. Merge whenever you like, but release the client first.

Bug

StreamBlockScanDataShort and StreamComputeIndex did not check whether a height was indexed. For a height the oracle has not indexed, dbpebble.GetBlockHashByHeight returns (nil, nil): pebble.ErrNotFound is swallowed and does not come back as an error. That covers heights above the tip, below the sync start height, and gaps. The streams then sent an OK message with an empty block hash, an empty compute index and no spent outputs.

A height above MaxUint32 was also cast to uint32, so it wrapped around to a low height. The stream then served that other block's hash under the requested height.

The Rust scanner treats such a message as an indexed block with no payments. It moves past the height and never finds the payments in it, so funds are lost without any error. The blindbit-testkit mutation check unindexed-ok-empty shows this loss end to end.

Fix

  • A new small helper, streamBlockHash, is used by both streams. It returns status.Errorf(codes.NotFound, "height %d is not indexed", height) in two cases:
    • the stored hash is empty
    • the height is above math.MaxUint32
  • The stream sends every indexed height before the first unindexed one, then ends with that status.
  • Database errors behave as before.
  • The MaxUint32 check also stops a request with end near MaxUint64 from looping forever.
  • internal/server/GRPC.md:
    • The streaming section now states the contract: the stream ends with NOT_FOUND naming the height, every message before it is valid, and clients should bound end by the GetInfo height.
    • The error-codes table now mentions the streams.

Tests

The new file internal/server/v2/stream_test.go uses an in-memory pebble store and a bufconn gRPC server. Blocks are indexed at H and H+2, which leaves a gap at H+1. Its helper names differ from those in #64's helpers_test.go.

Each case below runs against both streams:

Range Expected
[H, H] and [H+2, H+2] one message with the correct display-order hash, clean end
[H, H+2] exactly the H message, then NOT_FOUND naming H+1
[H-1, H] NOT_FOUND naming H-1, no messages
[H+3, H+3] NOT_FOUND, no messages
[H+2, H+3] the H+2 message, then NOT_FOUND naming H+3
[2^32+H, 2^32+H] NOT_FOUND; before the fix this aliased to block H

Red before the fix: with the unfixed service.go, 10 of the 14 subtests failed. Every unindexed case got OK messages with empty hashes. For example, [H, H+2] returned 3 messages with {501 []} in the middle, and 2^32+H returned block H's hash. After the fix, all 14 pass.

Verification

Command Result
go build ./... && go vet ./... && go test ./... build and vet OK; tests ok for internal/indexer, internal/server/v2 and internal/types
CGO_ENABLED=1 go build -tags libsecp256k1 ./... && CGO_ENABLED=1 go vet -tags libsecp256k1 ./... && CGO_ENABLED=1 go test -tags libsecp256k1 ./... same result

golangci-lint is not installed locally, so it was not run.

Overlap with #64

#64 (unmerged) makes the unary per-height RPCs return NOT_FOUND. It leaves the streams unchanged, and its GRPC.md says the streams do not use NOT_FOUND. This PR covers the streams.

Both PRs edit internal/server/GRPC.md, so whichever merges second needs a small GRPC.md rebase:

There is no code conflict: this PR's helper is streamBlockHash, and #64's is indexedBlockHash.

🤖 Generated with Claude Code

StreamBlockScanDataShort and StreamComputeIndex answered a height the
oracle has not indexed (above the tip, below the sync start, a gap, or a
height > MaxUint32 that aliased to an indexed one via uint32 truncation)
with an OK message carrying an empty block hash and no data. Clients read
that as an indexed block without payments and skip it, silently losing
funds.

Both streams now end with codes.NotFound naming the height at the first
unindexed height, after sending every preceding indexed block. Database
errors keep their existing behaviour. GRPC.md documents the contract and
advises bounding `end` by the GetInfo height.

Tests cover both streams over an in-memory pebble store with a gap, below
the first block, above the tip and beyond the uint32 range.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant