Skip to content

fix(blocksync): hand over to consensus at the tip despite unserved peer height claims - #1618

Merged
lklimek merged 38 commits into
devfrom
fix/blocksync-handover-at-tip
Nov 12, 2026
Merged

lklimek merged 38 commits into
devfrom
fix/blocksync-handover-at-tip

Conversation

@Claudius-Maginificent

@Claudius-Maginificent Claudius-Maginificent commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

TL;DR: A restarted node now returns from block sync to consensus once it has caught up with the chain, even if one of its peers reports a height it never delivers.

User story

As a validator operator, I want a restarted node to resume voting as soon as it has caught up, to keep my validator participating in consensus regardless of what individual peers report.

Scenario

Base flow

A validator restarts, downloads the blocks it missed through block sync, and then switches to consensus.

Actual behavior

Block sync waits until the node reaches the highest height any connected peer reports. Peer reports are not checked, so a peer reporting a much higher height that it never delivers keeps the node in block sync: the node follows the chain but does not vote or propose. Blocks arriving from other peers keep the existing stall checks from firing, and a peer dropped for failed requests comes back with a clean history on its next status message.

Expected behavior

Once the node's latest block is recent, the node switches to consensus within about two minutes, whatever heights peers report. A node that is genuinely behind keeps syncing.

Detailed discussion

What was done

Production changes are in internal/blocksync/synchronizer.go, reactor.go and peer_store.go; the block request scheduler and the p2p client are unchanged. Heights reported by peers are not verified, so the new rules rely only on the validator-signed time of the node's last applied block and on the node's own progress.

  • At the tip. The node is at the tip while its last applied block is less than 5 minutes old and no further ahead of the local clock than the chain's synchrony precision (read from the applier state, so block times never come from peer messages). After 2 minutes at the tip it hands over to consensus, whatever heights peers report; the 2 minutes start over whenever the last block has become 30 s more recent since they began, so a node still catching up finishes first. The catch-up target passed to consensus is the highest report within 10 blocks of the node.
  • No progress for 10 minutes. The node hands over when the wait began on a last block less than a day old. With an older block the previous rule applies (hand over only when no report is more than 10 blocks ahead), so a node far behind keeps syncing (consensus: after a block-sync handover while still behind, a validator proposes at historical heights and wedges itself permanently #1413).
  • Overdue next block. When no block has been applied for 60 s (the request deadline), the next block is requested once from every peer advertising it, so it does not depend on one peer's answer.
  • The existing rules (caught up with the highest report; nothing to fetch for 60 s) are unchanged.

What operators can expect (also in reactor.md):

  • A node that has caught up hands over within two to four minutes, whatever heights its peers report, provided a connected peer serves each new block when asked. Peers that delay blocks can stretch this to about 20 minutes, and a single block by about a minute.
  • On a chain that has stopped producing blocks, a restarted node hands over after 10 minutes without a block. If the chain had been stopped for more than a day when the node started waiting, the node keeps waiting while a peer reports the next height and a height more than 10 blocks above it.
  • A node may hand over up to five minutes of blocks behind (one or two blocks at a three-minute interval, a few hundred on a chain committing every second), only when it received blocks less than a quarter faster than the chain produced them. Consensus catch-up closes the rest.
  • A node less than a day behind whose peers all fail to deliver the next block for 10 minutes hands over where it is.
  • With only a few peers serving blocks, a node restarting far behind a chain that commits a block every second or two may need much longer to reach the tip; more peers that serve blocks help.
  • A clock minutes ahead of the validators', or behind by more than the synchrony precision, leaves only the other rules.
  • After a handover below a height a peer reported, the node may hold back its own proposals for up to 10 minutes; with a recent last block only reports within 10 blocks count.

Testing

  • internal/blocksync tests (fake clock) cover: peers reporting far heights that stay silent, answer NoBlockResponse, report a range above the node, answer just before the request deadline, deliver each block late, are dropped and re-added, or reconnect under a new node ID every minute; live, bursty (1–5 s), fast and halted chains (30 min to just under a day, and over a day); a node far behind (consensus: after a block-sync handover while still behind, a validator proposes at historical heights and wedges itself permanently #1413); restarts with a recent last block; clock skew both ways; the handover target on every exit; fetch volume (each block fetched once in steady sync, far behind, on a slow downlink and at the tip); peers faster than the chain but not by much. The handover tests failed before the change.
  • go test -race -tags=deadlock ./internal/blocksync/... ./internal/p2p/client/...: pass; new and handover tests pass repeated -race runs.
  • golangci-lint v2.15.0 with --new-from-rev=origin/dev: 0 issues.
  • Full suite and e2e left to CI.

Breaking changes

None. Block sync may hand over to consensus earlier in the situations described above.

Attribution

🤖 Co-authored by Claudius the Magnificent AI Agent

🤖 Generated with Claude Code

PR Hygiene · eaed8f0

  • Bots — thepastaclaw ✓
  • Self-review — not asked of a bot author
  • Build green
  • Approvals
    • files with no dedicated owner (docs/tenderdash-core/block-sync/reactor.md, internal/blocksync/ask_every_peer_test.go, internal/blocksync/handover_bursts_test.go and 9 more) — approved by lklimek

When every merge requirement is met, the PR Hygiene check passes. Reviewer limits do not block merging; other required GitHub checks and protections still apply.

…er height claims

Block sync considered the node caught up only once it reached the highest
height any peer reported. A peer that reported a much higher height and did
not deliver blocks kept that claim counted: the existing stall checks apply
only when no peer advertises the next height or the node is within 10 blocks
of the claim, blocks arriving from other peers kept resetting the stall timer,
and a peer dropped after failed requests was re-added with a clean history on
its next status. A restarted node could then stay in block sync instead of
returning to consensus.

Record, per peer ID, the lowest height the peer failed to serve (timeout or
other failed request, NoBlockResponse, or a block that fails to apply). The
record survives dropping and re-adding the peer, is cleared when a block that
peer supplied at or above that height is applied, and expires 10 minutes
after the last failure. Claims at or above it stop counting.

Judge whether the node is at the tip from its last applied block time, which
the validators commit to: once that time has stayed within five minutes of the
local clock (and no further ahead than the synchrony bounds allow) for two
minutes, and no counted claim covers the next height, hand over. If the last
block is older, hand over after 10 minutes without progress when no counted
claim covers the next height. The consensus catch-up target leaves out
discounted claims. Bounds and remaining cases are documented in
docs/tenderdash-core/block-sync/reactor.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor
Pre-merge checks: Passed 1 · Inconclusive 1

Failed Failed checks: 1 inconclusive

Check name Status Explanation Resolution
Title check Pending This assessment has not completed. The check will be retried on the next pre-merge check run.
Passed Passed checks: 1 passed

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@thepastaclaw

thepastaclaw commented Nov 11, 2026 •

Copy link
Copy Markdown

✅ Final review complete — no blockers (commit eaed8f0) · triage: normal

@lklimek
lklimek marked this pull request as ready for review November 11, 2026 10:54
@lklimek lklimek added this to the v2.0 milestone Nov 11, 2026
@lklimek lklimek added the priority: high High priority — address soon label Nov 11, 2026
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post /skip-bots to proceed without them; blocking findings still need addressing.
Full checklist in the description.

@github-actions github-actions Bot added the waiting-bots Waiting for the review bots to report on this head label Nov 11, 2026
lklimek and others added 10 commits November 11, 2026 11:34
…claims

Node IDs cost nothing, so a per-peer record of unserved heights cannot keep
a far height claim from holding a node in block sync: a claim made under a
new ID every minute, or by a peer that supplies each block as the network
commits it, never stopped counting.

Base the handover on what peers cannot fake instead:

- At the tip: the node is at the tip while its last applied block is no
  older than the chain took to commit its last maxCatchupGap blocks (at
  least 70 s, at most 5 min) and no further ahead of the local clock than
  the chain's current synchrony precision. After two minutes there it hands
  over whatever peers claim; the catch-up target leaves out claims more than
  maxCatchupGap above it.
- 10-minute backstop: far behind, only a peer that supplied a block the node
  applied, and has not failed a request since, keeps the node syncing. A new
  peer has no such credit.

The block time comes from the applier state rather than the block response.
The unserved-height map and its expiry are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nder rotating peer IDs

Fails on f0bf55d: a fresh peer ID whose far claim has not yet failed counts
as credible, so a peer rotating IDs faster than syncTimeout keeps every
handover path (tip grace, 10-minute backstop) from firing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Until block sync has applied maxCatchupGap blocks since it started, the at-tip
window fell back to tipRecency, so a node restarting minutes behind a fast
chain was taken for one at the tip when its source supplied fewer than that
many blocks within the grace period. Scale the span of the blocks applied so
far up to maxCatchupGap instead; with no block applied yet, tipRecency alone
still applies.

Also test that starting block sync seeds the last block time from the stored
state and that the synchrony precision follows the current state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…und in review

Both tests are expected to fail on 7451f76:
- a far claim from a peer that backed nothing keeps a stalled node in
  block sync while another, near peer has backed its claim
- a pushed-back height an idle peer serves waits behind a higher height
  only a busy peer claims

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ims only

The backstop took how far behind the node is from any servable claim and the
credit from any peer that backed its claim, so a far claim nobody backed,
next to a near peer that had, kept a stalled node in block sync. Take the
distance from claims of peers that supplied a block applied since they last
failed a request; with none, the node is not known to be behind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er one

The job producer took the next height and waited until some peer could take
it. When only busy or cooling peers claimed that height, a lower height
queued for a retry - typically the one block sync waits to apply - was not
requested, even with an idle peer advertising it. Put the higher height back
and request the lower one first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…did not

Peer selection took a due recovery probe first and otherwise the first
eligible peer, so new peers, which come with no history, could take turns
holding the height block sync waits for, each until its request timed out.
Ask a peer that supplied a block applied since it last failed a request
before any other; among the rest the order is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The handover loop read the synchrony precision from the applier every
second, and the applier holds its lock for the whole time a block is
executed, so a slow block kept the loop from noticing cancellation. Record
the precision with the block time when a block is applied, seed it at start,
and read it from the stall snapshot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The two-minute bound holds once the node has reached the tip and while the
next block keeps reaching it within its at-tip window. Describe the request
order, how a peer delaying the next block keeps the node from the tip, with
one node ID or several, and how peers that report their status first can
cause the restart case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Phase 1 + Phase 2

The change adds useful failure evidence and an authenticated recency signal, but handover target accounting, recovery timing, and applied-state observation still contain liveness defects. Static verification at f0bf55d confirmed four blocking issues; no builds or tests were run locally. Current CI has build, lint, vet, and both e2e configurations passing, with test_abci_cli and several unit-test shards pending; govulncheck fails on an unrelated Go standard-library vulnerability.

🔴 4 blocking

Review provenance

Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: tenderdash-consensus-security); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: tenderdash-consensus-security); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: normal by gpt-6.1-sol (effort low) — The change adds intricate block-sync peer tracking and handover logic, but does not itself change consensus rules, cryptography, signature handling, peer-facing deserialization, funds movement, or storage migrations.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity quota lookup failed), glm-5.3-flash (not used above high effort; tier asks max)
  • Second Phase-2 pass: on this first full review every Phase-2 reviewer also ran a second, independent time, and the verifier weighed both sets
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — tenderdash-consensus-security (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — tenderdash-consensus-security (completed, effort high); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review (PastaClaw). How to work through them:
- Verify each finding against the code; fix only what is real, as narrowly as the finding.
- If a finding is wrong, decline it on its thread in one or two sentences; the bot answers once and stops.
- To defer one, link a tracking issue on its thread (or reply asking the bot to file one).
- Suggestions don't block the merge, but most are worth fixing in this PR; judge each on its merits.
- Keep the PR's scope; don't grow it to satisfy hardening suggestions.
- Carried items were raised in an earlier round: read the thread first, and don't re-fix what you already fixed, declined or deferred there. Disputed and deferred items are already handled and are not listed.

In `internal/blocksync/synchronizer.go`:
- [BLOCKING] internal/blocksync/synchronizer.go:479-482: Track discount-aware historical targets across every handover exit
  The two target calculations lose different necessary information: targetHeight retains every raw claim forever, while stopUnservedClaims returns only the currently connected peers' maxCountedHeight. For example, after an inflated claim has been observed, NoBlockResponse records the failure and makes its peer unservable at the next height. With no intervening blocks, stopNothingToFetch wins after 60 seconds, before the two-minute grace, and returns the inflated historical target. Timeout-driven peer removal can similarly make the caught-up branch return that target. SwitchToConsensus then arms catchupTracker and suppresses this validator's proposals for up to another ten minutes, despite the recorded failure; voting remains enabled.

  Conversely, when stopUnservedClaims wins, an unfailed higher peer that disconnected earlier disappears from maxCountedHeight. This discards legitimate observed catch-up targets and can prevent proposal suppression from being armed while the node is still behind. TestWaitForSyncRetainsObservedTarget establishes the existing retention requirement, but does not exercise this new exit. Retain observed claims with peer attribution, apply the current failure discount to those historical claims, and use the resulting target consistently across normal handover exits. Add returned-target regressions for NoBlockResponse, failure-driven removal, and an unfailed disconnected higher peer.
- [BLOCKING] internal/blocksync/synchronizer.go:668-669: Give a newly failed productive source time to recover
  atTipFor measures how long block timestamps have looked recent, including time spent successfully downloading. It does not measure how long the source has failed. A node can therefore download productively for more than two minutes while remaining several minutes behind a fast chain, then lose its sole counted source's credibility on one transient fetch failure. This branch ends block sync on the next tick, even when a block was applied shortly beforehand and a retry would succeed. It runs before the stall checks and gives the source neither the documented cooldown nor a recovery probe. Because ordinary block-sync handover is irreversible, the remaining gap is left to much slower consensus catch-up. Tie recovery eligibility to the loss of service or otherwise preserve a recovery opportunity for a recently productive source without letting unrelated tip traffic refresh a silent peer indefinitely. Add a regression with recent, nonzero block timestamps: flakySource currently supplies zero timestamps, which disables this branch in the relevant keep-syncing tests.
- [BLOCKING] internal/blocksync/synchronizer.go:253-256: Refresh synchrony bounds after committed parameter updates
  synchrony is copied only during OnStart, although downloaded blocks can change Precision and MessageDelay through CurrentRoundState.populateConsensusParams and UpdateState. advance updates lastBlockTime but leaves isRecent using the startup bounds. For example, after a committed precision increase to 60 seconds, a node whose clock trails validators by 30 seconds is within the new bounds but outside the cached default allowance of 12.505 seconds. On a live chain producing blocks every 30 seconds, each new timestamp resets atTipSince before it can reach two minutes. Honest block applications also keep resetting lastAdvance, so an unserved inflated claim can still prevent handover indefinitely. A parameter decrease has the opposite problem: the future-time safeguard remains too permissive. Refresh the effective synchrony parameters from the applied state alongside the block-time observation, and read both consistently when deciding recency. Cover a parameter update during block sync.
- [BLOCKING] internal/blocksync/synchronizer.go:757-761: Credit the applied response independently of its pending entry
  applyBlock copies a pending response, releases s.mtx, and calls the potentially long-running applier.Apply. A concurrent PeerStatusDown can then invoke RemovePeer, whose dropPeer deletes that response from pendingToApply. If application succeeds, advance no longer finds the entry and skips both RecordServed and lastBlockTime, despite incrementing the height and refreshing lastAdvance. The supplier can remain discounted after reconnecting even though its block was verified and applied, and the recency signal can retain an old timestamp while current blocks are successfully applied. Repeated disconnections during application can therefore prevent the new tip rule from activating while progress continually defeats the stall timeout. Pass the successfully applied BlockResponse into advance and update its supplier credit and timestamp under s.mtx regardless of whether the queue entry survived. Add a regression that removes the supplier while Apply is in flight.
Out-of-scope follow-up suggestions (1)

These are valid observations, but they are outside this PR's scope and should be handled in separate issues or author/maintainer-requested PRs rather than blocking this review.

  • Update the Go toolchain for the reported HTTP server vulnerability — The failed govulncheck log identifies GO-2026-6859, an HTTP/1 server crash involving a MaxBytesReader header race, in net/http@go1.27.3, fixed in go1.27.4. The repository uses MaxBytesReader in rpc/jsonrpc/server/http_server.go, but this PR changes neither that code nor the toolchain.
    • Follow-up: Open a separate toolchain-update PR using Go 1.27.4 or a later patched release for builds and CI, then rerun govulncheck and RPC validation.

Comment thread internal/blocksync/synchronizer.go Resolved
Comment thread internal/blocksync/synchronizer.go Resolved
Comment thread internal/blocksync/synchronizer.go Resolved
Comment thread internal/blocksync/synchronizer.go Resolved
@github-actions github-actions Bot added waiting-self-review Waiting for the author to post /self-reviewed and removed waiting-bots Waiting for the review bots to report on this head labels Nov 11, 2026
lklimek and others added 9 commits November 11, 2026 12:19
… for it fails

A retry of the height block sync waits to apply went to one peer at a time,
the best ranked first, so peers listed first - or due a recovery probe - could
each hold it until its request timed out while a peer able to serve it waited
behind them. Nothing was applied meanwhile, no peer earned credit, and after
10 minutes the stall backstop handed a node far behind over to consensus.
Once a request for that height fails, ask each other peer advertising it once,
alongside the retry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hile

The synchronizer lock is released while a block is applied, so the peer that
supplied it can be removed, and its pending response dropped, in the
meantime. The credit for the block was looked up from that response and so
was lost. Pass the supplier of the applied response to advance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…at the tip

Only the at-tip handover left claims out of reach of the tip out of the
target, and only those of peers still connected. A node at the tip handing
over because no peer could serve the next block, or because the far
claimant had gone, passed the far claim on and held back its own proposals
for up to 10 minutes, while a near peer that left was dropped from the
target. At the tip, every handover now targets the highest claim seen within
maxCatchupGap of the node.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… then

A node minutes behind a fast chain, with recent block times, does not hand
over when its only source times out a request now and then: the at-tip window
follows the chain's pace, not the source's credit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e target at the tip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…iling peers

Two tests that state the intended behaviour and fail on 4c57d66:

- TestWaitForSyncHandsOverBehindPeersSupplyingEachBlockLate: on a chain
  committing a block every three minutes, peers that report first after a
  restart and supply each block about three minutes late are asked for every
  block, the prompt peer never, and the node stays in block sync.
- TestWaitForSyncKeepsSyncingFarBehindPeersThatFailEveryRequest: a node far
  behind whose peers fail every request for ten minutes hands over to
  consensus (#1413).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WunYYgi2uMsQDcBcKAwLdt
… block is old

Taking the backstop's distance from backed claims alone let a node far back
in the chain's history hand over to consensus once its peers had failed
every request for 10 minutes (#1413). When the last applied
block is more than an hour old, any servable claim within reach counts again,
as before; the distance from backed claims applies to a node restarted while
its chain is stopped.

Also adapt the late-supply reproduction to the current advance signature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…old block

Both tests moved the clock past the backstop in one jump from a recent last
block, which skips the at-tip rule they would otherwise meet first. Start
them from an old last block and step the clock a second at a time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lklimek and others added 9 commits November 11, 2026 13:25
…nanswered

A peer answering each request just inside the deadline could hold the height
to apply for up to a minute without failing, which on a chain committing a
block every few seconds kept the node from ever counting as at the tip.

The generator now records, per height, when it was first requested, of whom,
and whether a request for it failed. Once the height to apply has failed, or
has gone unanswered for serviceGrace, it is asked of every peer that
advertises it. The generator reads the height to apply from the synchronizer
also while it waits for a peer inside nextJob, so a height that became the one
to apply meanwhile is covered. A height nextJob took and could not request
returns as the next new height rather than as a retry, a peer answering that
it has no such block has the height asked of one other peer only, and a
failed request for a block applied meanwhile is not queued again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s asked

With the height to apply asked of every peer after serviceGrace, block sync
at the tip no longer lags by a whole request deadline. Measured with one peer
answering just inside the deadline and a prompt one, the last applied block
reached 21-25 s of age on chains committing a block every 1-4 s. Set tipLag
to a status round plus three times serviceGrace (40 s) instead of 70 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
State the at-tip condition as applying each block before the previous one
leaves the at-tip window, the 40-second floor of that window, the 10-second
wait before every peer is asked for the next block, and the one-day limit
past which a stopped chain's restarted node keeps waiting behind far claims.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n a chain committing in bursts

Three tests that state the intended behaviour and fail on 4280100; all
pass with the generator and constants of 1ac0aca:

- TestBlockSyncFarBehindFetchesEachBlockOnce: twelve peers that each need
  a second to send a block, so a full queue of twenty takes twenty seconds.
  600 blocks take 6m5s and 4273 block responses instead of 3m21s and 600,
  because a height first requested more than serviceGrace ago is asked of
  every peer as soon as it becomes the one to apply.
- TestBlockSyncFarBehindOnASharedDownlink: the same with a downlink of four
  blocks a second in all: 6m53s and 1062 responses instead of 3m21s and 600.
- TestWaitForSyncHandsOverAtTheTipOfAChainCommittingInBursts: with prompt
  peers and a far range nobody is asked for, a chain committing 12 blocks
  three seconds apart and then none for 50 seconds keeps the node in block
  sync: the pause is longer than the 40-second at-tip window and shorter
  than the 60 seconds after which a node with nothing to fetch hands over.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WunYYgi2uMsQDcBcKAwLdt
…ch stall

With a 40-second floor, a pause in block production of 40 to 60 seconds after
a run of fast blocks ended the two minutes at the tip before the rule for a
node with nothing to fetch could apply, so a far claim kept a node at the tip
in block sync. Derive the floor from that stall plus a status round (70 s),
so a pause ends block sync by one rule or the other.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ecomes one

Heights requested ahead wait behind others in a peer's queue, so counting the
10-second wait from their first request had a node far behind ask every peer
for each block when a peer needed more than half a second per block. Count it
from the last request for the height or from when it became the one to apply,
whichever is later. Ask the others at once when every connected peer asked for
it is on probation for holding requests too long, so a slow peer holding
several heights in a row delays each by no extra wait.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Describe the 70-second floor of the at-tip window and why, what makes the
next block go to every peer that advertises it, from when the 10-second wait
counts, and that a "no such block" answer sends it to one other peer only.
Restore the GetStatus comment and update a test comment that described the
earlier generator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eer for an overdue block

Replace the per-peer bookkeeping the handover rules had grown with two
rules that use nothing a peer reports about itself:

- the last applied block has been at most tipRecency (5 min) old for
  tipGrace (2 min): hand over whatever heights peers report;
- no block for maxSyncStall (10 min) on a block less than staleBlockAge
  (a day) old when the stall began: hand over as well. Further back the
  distance to servable peer heights decides, as on dev (#1413).

To keep the first rule from depending on which peer holds the request for
the next block, a block 10 seconds overdue is requested from every peer
that reports it, once each, by the synchronizer itself.

Removed: the supplied-block credit and the peer ranking built on it, the
per-height request records in the job generator, the at-tip window that
followed the chain's pace, and client.RequestTimeout. The job generator,
the peer service and the p2p client are as on dev again.

A node may now hand over up to five minutes of blocks behind on any
chain; TestWaitForSyncOnAFastChainBehindTheTip pins that bound.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…xpect

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post /skip-bots to proceed without them; blocking findings still need addressing.
Full checklist in the description.

lklimek and others added 2 commits November 11, 2026 15:35
…peer only after a stall

The two minutes at the tip start over each time the last applied block has
got tipGain (30 s) younger, so a node applying blocks at least a quarter
faster than the chain commits them is not handed over on its way.

The next block is asked of every peer once nothing has been applied for
syncTimeout (60 s), the time a request may take, not after 10 seconds: on a
link carrying a block in more than 10 seconds every block was fetched from
every peer, and the requests queued behind them ran into their deadline.

The fetch-volume tests now ask the peers reserveNextBlockPeers returns, and
cover a slow downlink and peers faster than the chain.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…dovers hold back proposals

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post /skip-bots to proceed without them; blocking findings still need addressing.
Full checklist in the description.

…wrap a comment

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post /skip-bots to proceed without them; blocking findings still need addressing.
Full checklist in the description.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — Final validation — Phase 1 + Phase 2

At eaed8f0, the complete diff's block-time handover rules, progress-based grace resets, overdue-block fanout, and regression coverage reveal no actionable in-scope defects. Current CI passes the Linux build, both e2e scenarios, application tests, and documentation checks; Super linter, golangci-lint, go vet, govulncheck, and tests (00/01/02/03/05) fail during dependency installation with Ubuntu mirror DNS errors and exit code 100, unrelated to this change. This assessment is static; no local builds or tests were run.

🔴 0 blocking | 🟡 0 suggestion(s) | 💬 0 nitpick(s)

Review provenance

Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: tenderdash-consensus-security); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: tenderdash-consensus-security); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: normal by gpt-6.1-sol (effort low) — The intricate changes in internal/blocksync/synchronizer.go alter local sync handover and request scheduling, but do not change consensus rules, cryptographic validation, peer-facing deserialization, or storage migrations.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity quota lookup failed), glm-5.3-flash (not used above high effort; tier asks max)
  • Fresh final gate: an independent Phase-2 review ran after iterative findings were reconciled
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — tenderdash-consensus-security (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — tenderdash-consensus-security (completed, effort high); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify the current code and confirm that no unresolved issues remain.

No unresolved findings remain from the prior review on this head.

@github-actions github-actions Bot removed the waiting-bots Waiting for the review bots to report on this head label Nov 11, 2026
@github-actions

Copy link
Copy Markdown

Ready for review — files with no dedicated owner: lklimek or shumkov.
Full checklist in the description.

@github-actions github-actions Bot added the ready-for-human PR Hygiene: bots done, author self-reviewed, awaiting human review label Nov 12, 2026
@github-actions
github-actions Bot requested review from lklimek and shumkov November 12, 2026 02:38
@lklimek
lklimek enabled auto-merge (squash) November 12, 2026 18:42
@github-actions

Copy link
Copy Markdown

Policy satisfied — this can merge.
Full checklist in the description.

@github-actions github-actions Bot removed the ready-for-human PR Hygiene: bots done, author self-reviewed, awaiting human review label Nov 12, 2026
@lklimek
lklimek merged commit 4e15907 into dev Nov 12, 2026
34 of 43 checks passed
@lklimek
lklimek deleted the fix/blocksync-handover-at-tip branch November 12, 2026 18:44
lklimek added a commit that referenced this pull request Nov 12, 2026
…1621)

#1595 replaced the decoded block in the synchronizer's BlockResponse with
its committed bytes and a Height field. The handover tests from #1618,
merged after it, still set and read BlockResponse.Block, so the
blocksync test package no longer compiled on dev. They only need the
height, which addBlock reads from Height.

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>


Claude-Session: https://claude.ai/code/session_01WunYYgi2uMsQDcBcKAwLdt

Co-authored-by: Lukasz Klimek <842586+lklimek@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority: high High priority — address soon

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants