Repository navigation
fix: make initial sync work against modern Bitcoin Core - #63
Merged
Merged
Conversation
Three defects, each of which independently prevented a fresh oracle from completing an initial sync. All three were found by running the oracle against a Bitcoin Core v30.3 regtest node, not by reading code. 1. chaininfo.warnings could not be parsed. Bitcoin Core v25 and later return `warnings` as an array, while the struct declared it a string, so sync died immediately with "cannot unmarshal array into Go struct field ChainInfo.warnings of type string". This is the same change as the open pull request by bitsagarob; it is included here so the integration branch can sync at all, and can be dropped if that lands first. 2. Syncing a fresh database underflowed past genesis. SingleBlockPullAndHandle recurses to the previous block whenever the previous blockhash is absent from the database. Its doc comment says it is intended for reorgs rather than full syncs, but on an empty database the previous hash is never present, so it walked from the chain tip down to height 0 and then asked for height-1. The parameter is unsigned, so that wrapped to 4294967295, Core answered 400, and rest.go treats a non-200 as fatal. The recursion now stops at genesis and at the operator's configured sync_start_height, since descending below that indexes blocks they asked to skip. 3. A missing viper default silently disabled the whole indexing pipeline. vars.go declares MaxParallelTweakComputations = 2 as the intended default, but config.go overwrites it with viper.GetInt and, unlike max_parallel_requests, never registered a default for that key. A config file that omits it therefore resolved to 0, and SyncBlocks starts exactly that many block-handler goroutines. With none running, nothing drained the block channel, the writer channel closed immediately, and the sync returned success having indexed no blocks at all. That third failure is the dangerous one: a crash is loud, but an operator whose config omitted one optional-looking key got a success message, a zero exit code, and an oracle that served empty results forever. Distinguishing "no payments in this block" from "nothing was ever indexed" is impossible downstream, so the configuration is now validated and a parallelism below 1 is fatal rather than quietly doing no work. Verified on regtest against Core v30.3: a fresh `run` no longer underflows and indexes the chain; a fresh `sync` with a minimal config that omits the parallelism keys processes 105 blocks where it previously processed zero; and an explicit max_parallel_tweak_computations = 0 now exits non-zero with a message naming the key instead of reporting success. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Non-testkit work now lands on the integration branch first and is promoted to master only after review, but the build-test workflow only triggered on pull requests into master. Every pull request into integration therefore ran no CI at all, which is precisely where the checks are wanted, since integration is the branch that accumulates unreviewed work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three defects, each of which independently prevents a fresh oracle from completing
an initial sync. All three were found by running the oracle against a Bitcoin
Core v30.3 regtest node, not by reading code.
Based on
integration, per the rule that non-testkit work lands there rather thanon
master.1.
chaininfo.warningscannot be parsedCore v25+ returns
warningsas an array; the struct declared it astring:Same change as the open pull request from bitsagarob (
Warnings→json.RawMessage). Included sointegrationcan sync at all — drop this hunk ifthat PR lands first; the credit is theirs.
2. Fresh sync underflows past genesis
SingleBlockPullAndHandlerecurses to the previous block whenever the previousblockhash is missing from the database. Its own doc comment says it is "intended to
handle reorgs not full chain syncs" — but on an empty database the previous hash is
never present, so it walked from the tip to height 0 and then asked for height −1.
The parameter is
uint32, so that wrapped:rest.gotreats any non-200 as fatal, so the process died. The recursion now stopsat genesis and at the configured
sync_start_height, since descending below thatindexes blocks the operator asked to skip.
3. A missing default silently disabled the indexing pipeline
vars.godeclaresMaxParallelTweakComputations = 2as the intended default, butconfig.gooverwrites it withviper.GetInt(...)and — unlikemax_parallel_requests— never registered aSetDefaultfor that key. A config thatomits it resolves to 0, and
SyncBlocksstarts exactly that many block-handlergoroutines. With none running, nothing drains the block channel, the writer channel
closes immediately, and the sync returns success having indexed nothing:
This is the dangerous one. A crash is loud. An operator whose config omitted one
optional-looking key got a success message, a zero exit code, and an oracle that
served empty results forever — and nothing downstream can tell "no payments in this
block" from "nothing was ever indexed". So the configuration is now validated, and a
parallelism below 1 is fatal rather than quietly doing no work.
Verification
Run on regtest against Core v30.3, before and after:
run4294967295sync, config omitting the parallelism keys/tweaks/102populatedmax_parallel_tweak_computations = 0must be at least 1, got 0go build ./...,go vet ./...,go test ./...andgofmtare all clean.Not covered here
The index-mode flags remain inert — they populate
GetInfoand gate neither indexingnor serving, and
?dustLimit=is ignored. That is tracked separately as SNB-543 andis deliberately out of scope for this fix.
🤖 Generated with Claude Code
Note
Medium Risk
Touches the initial sync and block-pull path plus startup config validation; changes are targeted bug fixes but affect core indexing behavior on fresh deployments.
Overview
Fixes three independent bugs that blocked a fresh oracle from completing initial sync against recent Bitcoin Core (e.g. v30.3).
REST chaininfo parsing:
ChainInfo.warningsis nowjson.RawMessageso Core v25+ can returnwarningsas an array without failing JSON decode.Block pull recursion:
SingleBlockPullAndHandleno longer walks past genesis on an empty DB. Recursion into the previous block only happens whenheight > 0andheight > sync_start_height, avoidinguint32underflow to height4294967295and fatal REST 400s.Config defaults and validation: Registers a viper default for
max_parallel_tweak_computations(omitted keys were resolving to 0 workers and a “successful” sync with zero blocks indexed). Startup now fatals ifmax_parallel_tweak_computationsormax_parallel_requestsis below 1.Reviewed by Cursor Bugbot for commit 7d26a86. Configure here.