Repository navigation
feat(consensus)!: recover a beacon from prevotes and sign precommits over the randomness-applied app hash - #1583
Draft
QuantumExplorer wants to merge 41 commits into
Draft
QuantumExplorer wants to merge 41 commits into
QuantumExplorer wants to merge 41 commits into
Conversation
…elds
Add a new ABCI method, ApplyBlockRandomness, for same-block randomness.
Tenderdash will call it at consensus version 2 with the random beacon of a
block and round (the BLS threshold signature recovered from its prevotes),
and the application returns the app hash the precommits sign.
- proto: RequestApplyBlockRandomness {hash, height, round, beacon} and
ResponseApplyBlockRandomness {app_hash} in the Request/Response oneofs and
the ABCI service; CommitInfo.beacon and CommitInfo.app_hash;
Commit.beacon and Commit.app_hash.
- Go: Application interface, socket, gRPC, routed and local clients, socket
server, proxy client with timing metrics, regenerated mocks.
- BaseApplication returns ErrApplyBlockRandomnessNotImplemented: an empty
app hash must never pass as an answer.
- kvstore example: returns the app hash of the processed block unchanged.
- types.Commit carries Beacon and AppHash through protobuf and CommitInfo.
MaxCommitOverheadBytes grows by 132 bytes (96+2 for the beacon, 32+2 for
the app hash).
Nothing calls the method yet, and the new fields stay empty, so behaviour is
unchanged apart from the larger commit overhead reserved by the mempool size
pre-check.
Refs dashpay/platform#4682
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…over the randomness-applied app hash At consensus version 2 the threshold signature recovered from a block's prevotes is the block's random beacon. Validators hand it to the application with ApplyBlockRandomness, and precommits and the commit sign the app hash it returns. Nothing changes before the application sets consensus version 2. - params: CONSENSUS_VERSION_2, VersionParams.RandomBeacon(), unknown consensus versions refused; proposer selection as in version 1. - vote set: WithRandomBeacon makes a prevote majority for a block count only once its threshold signature is recovered, since the majority gate can be below the recovery threshold; ThresholdBlockSignature exposes the beacon. - precommit signing: ApplyBlockRandomness with the round's beacon, state ID over the returned app hash, then ExtendVote, then sign; the state ID is cached per height, round and block so a node never signs two. - commit: carries the beacon and the final app hash; Commit.VerifyRandomBeacon checks the beacon against the quorum key over the prevote sign item and the state ID against the header with that app hash. Commits built from own precommits wait for the beacon or a peer's commit. Pre-version-2 peer commits have both fields dropped; parked commit keys include them. - BlockID.SameBlock for commit-vs-proposal comparisons; FinalizeBlock uses the commit's BlockID, applies the commit's beacon before the application's FinalizeBlock and stops on an app hash mismatch; state.LastAppHash is the commit's app hash; new State.LastBlockBeacon, which the next block's LastCommit must match. - block sync, replay, peer commits: randomness applied after ProcessProposal and before the commit's extension check, stopping on a mismatch; rollback restores the commit's BlockID, app hash and beacon. - light client and state sync use the app hash the commit signs; state sync restores LastAppHash and LastBlockBeacon from the snapshot commit at version 2. - kvstore example: toy translation sha256(app hash || beacon). Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the
|
…vation of the random beacon - A block locked in round 0 and voted again in round 1 gets the round 1 beacon: the round 1 precommit signs another final state ID, and the block commits at round 1 with that beacon. - The final state ID a validator signs is cached per block and round: a second precommit does not ask the application again, even if it would answer differently. - A node outside the validator set finalizes every block from peers' commits at consensus version 2 and reaches the validators' beacon and final app hash. - The rotate e2e network activates consensus version 2 at height 1010, so state sync, block sync and restarts run with it before the network moves back to versions 1 and 0. TestApp_Hash and TestBlock_Header compare with the app hash the commit signs instead of the header's. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
🕓 Review not started yet because this PR is a draft.
Commit 1b17130. Normal review starts when eligible; priority review starts as soon as a slot is available. |
3 of 4 tasks
TestRandomBeaconRelockNewBeacon copied the validator set with GetValidatorSet while the consensus goroutine was entering round 1, which moves the set's proposer index in place. Copy the set before consensus starts: the test only reads its quorum type, hash and threshold key, which do not change within the height. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Commit.beacon and Commit.app_hash are covered neither by the commit hash nor by its threshold signature, so a peer relaying a validly signed commit could fill them with any bytes. Before this fix a block sync peer could pad the beacon of an honest commit: the node verified the signature, stored the padded commit as its seen commit and passed it to FinalizeBlock. Over gRPC ABCI a beacon above 4 MB made FinalizeBlock fail and the node crash on every restart, and a beacon larger than Block.MaxBytes left it unable to propose. Commit.ValidateBasic now refuses a commit unless both fields are empty or exactly 96 and 32 bytes, whatever the consensus version. Every path that accepts a commit on its own decodes it through CommitFromProto or validates it with ValidateBasic (block sync responses, consensus Commit messages, light blocks, stored commits), so such a commit is refused before it is stored or passed on. A block's LastCommit is not refused for these fields: releases before them skip them as unknown, so refusing such a block would split an upgraded node from them. BlockFromProto decodes it without the malformed fields instead, so they are never read, kept or passed on. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…elds Releases before Commit.beacon and Commit.app_hash (fields 7 and 8) skip those fields whatever their wire type. The generated decoder of this release instead refuses the whole block when its LastCommit carries one with a wire type other than bytes. In a network still running older releases, a byzantine proposer could therefore get such a block committed by the older validators while upgraded nodes could not decode it, and an older node that stored it would panic loading it after upgrading. types.UnmarshalBlockProto decodes block bytes as before and, only when that fails, drops the mistyped occurrences of those two fields from the block's LastCommit and decodes again, so an upgraded node reads every block an older one does. Consensus block part assembly and BlockStore.LoadBlock use it. google.golang.org/protobuf becomes a direct dependency for its wire format parser. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The example application looked up the state for ApplyBlockRandomness by height and round only. A node that executed two blocks of one round, as with an equivocating proposer whose second block the network commits after a later round was processed, failed with "more than one proposal executed", which the spec makes fatal, and a block that was not processed silently got another block's app hash. ProcessProposal now records which state each block hash left at each round, and ApplyBlockRandomness answers for the block the request names, refusing one that was not processed. Without a hash it still uses the only state of that height and round. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…andom-beacon Bring in the review fixes of the ABCI PR: commits with malformed beacon or app hash sizes are refused, blocks whose LastCommit carries mistyped beacon fields still decode, and the kvstore picks the ApplyBlockRandomness state by block hash. The kvstore equivocation test now expects the toy translation this branch adds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cessProposal At consensus version 2, RequestFinalizeBlock.commit carried the commit's beacon and app hash, but local_last_commit and proposed_last_commit of the next height's PrepareProposal and ProcessProposal always carried them empty, although CommitInfo documents both as set from consensus version 2. buildLastCommitInfo now sets them at version 2, from the state, which holds the values the last block was finalized with. The proposer's copy in block.LastCommit is checked against them only when the block is validated, after ProcessProposal, so it is never passed to the application. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…covers Commit.app_hash is not covered by the commit's signature. Before consensus version 2 a proposer could fill a block's LastCommit with a 32-byte app hash that binds nothing; VerifyAppHash, used by light blocks and state sync, then refused the height for good, so light clients could not verify it. VerifyAppHash now returns the header's app hash whenever the signed state ID is the header's, and requires the commit's app hash to bind the state ID only otherwise. VerifyRandomBeacon still requires the commit's own app hash to be the signed one. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ck commits The ApplyBlockRandomness contract left four things open: - Vote extensions: Tenderdash calls ApplyBlockRandomness before ExtendVote, but other processes verify a Precommit's extensions against the state ProcessProposal left, possibly before they recover the beacon. Extensions and their verification must not depend on the beacon. - What the randomness may change: only the state behind the returned app hash. Transaction results, events and validator set or consensus parameter updates come from ProcessProposal and are fixed by the header the Prevotes signed. - FinalizeBlock: from consensus version 2 the committed state is the candidate with the commit's beacon applied, and its root must equal commit.app_hash, so an application in same-block execution mode reads the randomness fields as well as the hash. - When it is called: on every path that finalizes a block from a commit, the node's own included; the proto said only commits the node did not assemble itself. The ABCI call grammar gains the method, and the CommitInfo table links VoteExtension to its proto instead of a missing anchor. Regenerated types.pb.go and api.md (comments only). Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ABCI interface gained the ApplyBlockRandomness method and the CommitInfo beacon and app_hash fields. RELEASES.md makes interface changes a minor bump, and v1.8.0 to v1.8.2 already report 1.4.0, so an application built against these protos could not tell from RequestInfo.abci_version that a Tenderdash without them is too old. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… reserve MaxCommitOverheadBytes had grown by the 132 bytes of a commit's beacon and app hash, but its only runtime use is the mempool's per-transaction size pre-check, and commits carry those fields only from consensus version 2. Before that, every chain lost 132 bytes of transaction size at CheckTx, and one with Block.MaxBytes between 1129 and 1260 refused every transaction while its proposals still had room. MaxCommitOverheadBytes is back to the commit without the fields, and the new MaxCommitRandomnessBytes holds them, for the consensus version 2 check to add. Commit.String, StringIndented and its log fields now show the beacon and the app hash, so two commits that differ only there can be told apart. TestCommitBeaconFields encodes and decodes Commit and CommitInfo on the wire and compares whole structs. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TestRouting routes ApplyBlockRandomness to its configured client and checks the response type, and TestApplyBlockRandomnessMetrics checks that the proxy client records the call under apply_block_randomness, for success and failure. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… consensus version 2 The mempool's per-transaction size pre-check reserves room for the next block's last commit. From consensus version 2 that commit also carries its beacon and app hash, so TxPreCheckForState now subtracts MaxCommitRandomnessBytes when the consensus params use the random beacon, and only then. Refs dashpay/platform#4682 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5 of 9 tasks
Brings in the source-derived consensus specification (#1562). Conflict in spec/abci++/abci++_tmint_expected_behavior.md: v1.8-dev drops the Commit call from the ABCI grammar, and this branch adds check-commit and apply-randomness. The resolution keeps both, and notes that from consensus version 2 the state FinalizeBlock persists has the root ApplyBlockRandomness returned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hen dropping mistyped commit fields The fallback in UnmarshalBlockProto drops a LastCommit's beacon and app hash fields that do not have the bytes wire type, so an upgraded node decodes every block an older one does. It split the bytes with protowire, which refuses encodings the generated decoder of older releases skips: a group closed by an end-group of another field number, and a ten-byte varint whose last byte overflows 64 bits. A proposer could then send a block that older validators accept and upgraded ones cannot decode. The scanner now reads tags, varints and skipped fields with the rules of the code gogoproto generates. The test checks each case against gogotypes.Empty, whose generated decoder skips every field as older releases skip fields 7 and 8, both for blocks it accepts and for blocks it still refuses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…quests buildLastCommitInfo built RequestPrepareProposal.LocalLastCommit and RequestProcessProposal.ProposedLastCommit field by field and left out the commit's beacon and app hash. It now keeps its initial-height guard and delegates to Commit.ToCommitInfo. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the source-derived consensus specification (#1562). Conflict in spec/abci++/abci++_tmint_expected_behavior.md: v1.8-dev drops the Commit call from the ABCI grammar, and this branch adds check-commit and apply-randomness. The resolution keeps both, and notes that from consensus version 2 the state FinalizeBlock persists has the root ApplyBlockRandomness returned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…andom-beacon Brings in the decoder fix and the proposal-side CommitInfo change from review. Conflict in internal/state/execution.go: the base branch builds the proposal requests' last commit info with Commit.ToCommitInfo, which copies the proposer's beacon and app hash, and this branch takes them from the state the last block was finalized with. The resolution uses ToCommitInfo for the other fields and keeps the state's beacon and app hash, leaving both empty before consensus version 2. TestProposalRequestsCarryLastCommitRandomness now expects the state's values. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fallback scanner truncates the field number to 32 bits and reads a length into an int that may wrap, as the code gogoproto generates does; gosec flagged both as G115. The conversions must stay as they are, so they carry nolint comments with the reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…andom-beacon Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve the go.mod conflict with the dependency bumps on v1.8-dev: keep google.golang.org/protobuf as a direct requirement, which types/block_decode.go needs, at the version v1.8-dev now pins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…andom-beacon Bring in v1.8-dev's dependency bumps through PR 1, whose merge resolved the go.mod conflict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A peer can add a correctly sized beacon and app hash to a validly signed commit: neither is covered by the commit's hash or threshold signature, so the commit is accepted and becomes the next proposal's LastCommit. Releases before the beacon drop both fields when they decode a block, and block sync rebuilds the block's parts from what it decoded, so they could not block sync a block an honest proposer built from that commit. CreateProposalBlock now drops both fields from the commit it is given, through the new Commit.WithoutRandomBeacon, which copies the commit rather than changing it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…andom-beacon PR 1 now drops a relayed commit's beacon and app hash when it builds a proposal. Here that applies until the last block is committed with a beacon: from then on validateBlock checks both against the state, so the commit keeps them. Commit.WithoutRandomBeacon moves to types/block.go with PR 1, and TestProposalRequestsCarryLastCommitRandomness keeps this branch's state-sourced expectations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TestMaxProposalBlockSize gives CreateProposalBlock a commit with a beacon and an app hash, which the proposal now drops, so its LastCommit and the block are back to their sizes without them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…andom-beacon TestMaxProposalBlockSize sets the state's last block beacon here, so the proposal keeps the commit's beacon and app hash and the commit is still at its largest, with them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rsion 2 Neither field is covered by the commit hash or its threshold signature, and both are empty before consensus version 2, which this release does not define. A peer relaying a validly signed commit could still add correctly sized ones, which nodes stored as the seen commit, passed to FinalizeBlock and served on; a proposer could put them in its block's LastCommit, which reached ProcessProposal. Drop both where a commit is adopted (consensus Commit messages, block sync, state sync and its backfill) and from the last commit passed in proposal requests. One test now covers the proposer side, with a block that keeps the fields as a negative control for older releases' block sync, replacing the two overlapping ones; another covers block sync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fallback allocated a copy of the whole block and one buffer per LastCommit occurrence before it knew whether anything had to be dropped. It now allocates nothing until the first field it drops, then writes everything into one buffer: 1 allocation instead of about 1M on 2 MiB of mistyped occurrences, and none when nothing is dropped. Varints are read with gogoproto's proto.DecodeVarint, which matches the generated code, instead of a copy of it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tenderdash always sends the block hash, so the height and round fallback for requests without one served only tests. ProcessProposal now records every block under its round and hash, and the lookup has one path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- spec/core/data_structures.md, consensus.md, consensus-commit.md: the Commit fields Beacon and AppHash, their size rule, and how a block's LastCommit with malformed ones is read. - Requirement 10 and the ABCI overviews: ApplyBlockRandomness must not modify the committed state, since it can run for rounds that never commit. - Commit.app_hash and CommitInfo.app_hash: the commit signs it only through its state ID. - ApplyBlockRandomness: this release defines consensus versions 0 and 1 only, so it never calls it. - UPGRADING.md: the new method, BaseApplication's error, ABCI 1.5.0, the commit size rule and routing with the routed transport. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…andom-beacon PR 1 now drops a commit's beacon and app hash wherever a commit is adopted. Here that applies before consensus version 2 only: - TryAddCommitAction, the block sync applier and buildLastCommitInfo keep this branch's version-aware code; - state sync drops them from the snapshot commit unless the state holds its verified beacon, and backfill keeps them only when they verify against the height's validator set; - the docs no longer say this release never calls ApplyBlockRandomness. TestProposalRequestsCarryLastCommitRandomness keeps this branch's state-sourced expectations, next to PR 1's merged TestProposalDropsRelayedCommitRandomness. The kvstore's toy randomness looks its state up by block hash only, as in PR 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve the conflict in abci++_basic_concepts.md with the mdBook docs migration: keep the ApplyBlockRandomness bullet next to the renamed ExtendVote one, and say Tenderdash in the lines this branch added, as v1.8-dev now does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…andom-beacon Bring in v1.8-dev (mdBook docs migration, dependency and CI updates) through PR 1, whose merge resolved the docs conflict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dev moved the ABCI specs from spec/abci++ to spec/abci and shortened the proto comments so the contracts live in methods.md. This carries the ApplyBlockRandomness text to the new files, keeps dev's short comments on ExtendVote and FinalizeBlock with one randomness sentence each, and regenerates types.pb.go and api.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…andom-beacon Brings in the dev merge from the base PR. Drops the "never calls it" line from the relocated spec/abci/methods.md, as this branch did in the old file. dev's replay fix for historical quorums returned two values from replayBlock, which returns three here, so its error path and new test are updated to match. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing signs the beacon and app hash of a block's LastCommit, so a proposer can fill them with well-sized values. SaveBlock stored them as the canonical commit of the previous height, and LoadBlockCommit then served them through /commit and state sync. Drop them from that record. The block's parts keep them, as the encoding the next commit signs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…andom-beacon Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The base branch drops the beacon and app hash of block.LastCommit from the stored previous-height commit, since nothing signs them. From consensus version 2 that record has to keep them for /commit and state sync. Keep them only when they match the commit this node saved for that height, which holds for no pre-version-2 commit and for every block validateBlock accepted after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Basic explanation
When enough validators prevote for a block, their BLS signature shares combine into one threshold signature that nobody could compute in advance and that every node recovers identically. From consensus version 2, Tenderdash uses that signature as the block's random beacon:
ApplyBlockRandomness(added in feat(abci)!: add the ApplyBlockRandomness method and commit beacon fields #1582), which fills its random values and returns the final app hash;Before, for any block:
After, at consensus version 2:
Nothing changes until the application sets
VersionParams.consensus_version = 2(Dash Platform does it at its protocol upgrade), so upgraded nodes behave as today until then.TL;DR: At consensus version 2, recover a random beacon from each block's prevotes, let the application apply it, and make precommits and commits sign the resulting app hash.
User story
As a Dash Platform game developer, I want a random value set in the same block that asks for it, which nobody (the user, the proposer, or any small group of validators) can predict or pick, so that card rarities and payouts are fair.
Scenario
Base flow
A Platform block creates a document with a random property. Drive writes a placeholder while executing the block, so the header app hash commits to the placeholder. The value must come from something nobody can know before the block is voted on, and must be in the state the block commits.
Actual behavior
Validators already produce such a value, the threshold signature of the prevotes, but Tenderdash never recovers it, and precommits always sign the header's app hash, so the committed state cannot depend on anything decided during voting.
Expected behavior
From consensus version 2, the prevote majority's threshold signature is the beacon; the application turns placeholders into values with it before precommitting; the commit signs the final state and carries what any node needs to verify and reproduce it. Client proofs keep verifying against the commit's signed state ID, unchanged.
Detailed discussion
What was done
Activation
CONSENSUS_VERSION_2inVersionParams,VersionParams.RandomBeacon(), and validation of the consensus version (unknown values are now refused inValidateConsensusParams). Proposer selection at version 2 is the same as at version 1.Beacon recovery (
types/vote_set.go)WithRandomBeaconmakes a prevoteVoteSetrecover the threshold signature of a majority for a block and count the majority only once recovery succeeds, as precommits already do. The majority gate (QuorumVotingThresholdPower) can be below the power BLS recovery needs, so otherwise a polka could form before the beacon exists. Nil prevote majorities are not recovered.VoteSet.ThresholdBlockSignature()exposes the beacon per round.HeightVoteSetturns this on per height from the height's consensus params.Precommit signing (
internal/consensus/vote_signer.go)ApplyBlockRandomnesswith the round's beacon, thenvote.BlockID.StateID = hash(header StateID with the returned app hash), thenExtendVote, then sign. The state ID is cached per (height, round, block) so a node never signs two state IDs for one block and round (that is duplicate-vote evidence). An application error panics, as forExtendVote.Commit
Commit.VerifyRandomBeaconchecks the beacon against the quorum's threshold key over the prevote sign item rebuilt from the header and the commit round, andhash(StateID(header, commit.AppHash)) == commit.BlockID.StateID.Commit.VerifyAppHash/SignedHeader.CommittedAppHashgive the app hash a commit signs at any version: the header's when the signed state ID is over the header (an unsignedapp_hashin the commit is then ignored), otherwisecommit.AppHash, which must be the one the state ID binds.beacon/app_hash(not covered by its signature) are dropped. Parked-commit deduplication keys now include both fields.Same block, two state IDs
BlockID.SameBlock(hash and part set header). Commit-vs-proposal comparisons indropStaleProposaland the proposaler use it;verifyCommitBlockuses the beacon check at version 2.FinalizeBlockuses the commit'sBlockID(it becomesstate.LastBlockIDand the next header'sLastBlockID, and is sent inRequestFinalizeBlock.block_id). At version 2 it callsApplyBlockRandomnesswith the commit's beacon and round right before the ABCIFinalizeBlockand stops on a mismatch withcommit.AppHash(ErrAppHashMismatch).state.LastAppHashbecomescommit.AppHash.PrepareProposal.local_last_commitandProcessProposal.proposed_last_commitcarry the last block's beacon and final app hash at version 2, taken from the node's state (LastBlockBeacon,LastAppHash), the valuesFinalizeBlockreceived incommit. The proposer's copy inblock.LastCommitis checked against the state only when the block is validated, afterProcessProposal, so it is never passed to the application.State.LastBlockBeacon(state proto field 17). The next block'sLastCommitmust carry the same beacon and app hash: neither is covered by the commit hash, and a block carrying others would hand block sync and light clients an unverifiable commit for the previous height.Other paths
BlockID, and applies the randomness afterProcessProposaland before the commit's extension check; a mismatch stops the node.replayBlockapplies the stored commit's beacon afterProcessProposal(decided by the replayed height's own consensus params) and stops on a mismatch;syncStateAtgoes throughApplyBlock, which does the same. Rollback restoresLastBlockID,LastAppHashandLastBlockBeaconfrom the commit of the rolled-back height.ProcessProposaland before the extension check, the same order a validator uses (randomness, thenExtendVote).LightBlock.ValidateBasicaccepts a commit state ID overcommit.AppHash; query proofs verify against the committed app hash.LastAppHashand the verifiedLastBlockBeaconfrom the snapshot height's commit when that height used version 2.kvstore example
ApplyBlockRandomnessreturnssha256(app hash || beacon), so every version 2 block has two different app hashes;FinalizeBlockchecks and commits it.Findings on two open items
checkAppHashEqualsOneFromBlockreceives inreplayBlock: theProcessProposalresult (ucState.AppHash), not the application's committed root. Comparing it with the header's app hash stays correct at version 2. What did break was the next check inreplayBlocks, which compared the same pre-randomness hash withstate.LastAppHash; it now uses the commit's app hash.LastAppHash = header(H+1).AppHash: inherited from upstream Tendermint's next-block execution (P2P state provider, 2021), where header H+1 carried the result of block H. With same-block execution it is the state after H+1, one block ahead of the snapshot;AppHash(H)had already been adapted,State()had not. It has gone unnoticed because nothing readsLastAppHashbefore the node commits H+1, except the handshake check after a restart at exactly the snapshot height.Rollbackhas the same off-by-one. This PR fixes it only for version 2 snapshots, to keep pre-activation behaviour unchanged; fix(statesync): restore LastAppHash of the snapshot height, not the next one #1584 fixes it for earlier versions.Testing
New targeted tests:
TestRandomBeaconConsensusVersion2: a validator at version 2 signs its prevote over the header state ID and its precommit over the final one, callsApplyBlockRandomnessbeforeExtendVote, and commits a beacon and app hash that verify; the state and the next header carry the commit'sBlockID, and the next block validates and commits.TestReactorRandomBeaconConsensusVersion2: four validators over the p2p test network, where each precommit needs the others' prevotes recovered; all nodes commit the same beacon and final app hash for every height.TestReactorRandomBeaconNonValidator: a node outside the validator set, which receives no votes, finalizes every block from peers' commits, applying the beacon itself, and reaches the validators' beacon and final app hash.TestRandomBeaconRelockNewBeacon: a block locked in round 0 and voted again in round 1 gets the round 1 beacon, so the round 1 precommit signs another final state ID, and the block commits at round 1 with that beacon.TestVoteSignerFinalStateIDCached: the final state ID is computed once per block and round; a second precommit does not ask the application again, even if it would now answer differently.TestLastCommitInfoRandomBeacon: at version 2 the next height'sPrepareProposalandProcessProposalget the state's beacon and final app hash in their last commit info, even when the proposer'sLastCommitcopy carries others; before version 2 both stay empty.TestVerifyAppHashIgnoresUnsignedAppHash: a commit whose state ID is over the header gives the header's app hash even when it carries another 32-byteapp_hash, whileVerifyRandomBeaconstill refuses it.FinalizeBlockrefusing a diverged app hash before the application sees the block, block sync refusing a bad beacon and stopping on divergence, replay order and divergence, rollback, and state sync's restored state.Regression runs with
-runover the touched areas of./internal/consensus/...,./internal/state,./internal/blocksync,./internal/statesync,./light/...,./types,./abci/...and./node(block size) pass.make proto-gen,make mockeryandmake lint(new findings againstv1.8-dev) are clean.End to end: the CI
rotatenetwork now activates consensus version 2 at height 1010, so its nodes state sync (from 1030 and 1035), block sync and restart at version 2, and the network then moves back to versions 1 and 0 at the existing heights.TestApp_HashandTestBlock_Headercompare with the app hash the commit signs, which is the header's before version 2. The e2e network was not run locally (no Docker here); CI runs it.Breaking changes
ApplyBlockRandomness.ValidateConsensusParamsrefuses unknown consensus versions (anything but 0, 1, 2).types.MaxCommitRandomnessBytes(132 bytes) more for the last commit's beacon and app hash (TestTxFilterRandomBeacon).sm.ExecutorgainsApplyBlockRandomness;StategainsLastBlockBeacon(new state proto field, empty in existing stores).Prior work
ApplyBlockRandomnessABCI method and commit fields), which should merge first. This PR targetsv1.8-dev, so its diff also shows feat(abci)!: add the ApplyBlockRandomness method and commit beacon fields #1582's commits; this PR's own changes are the commits from 90884b8 on, except the merges of feat(abci)!: add the ApplyBlockRandomness method and commit beacon fields #1582's branch (c1a2caf brought in feat(abci)!: add the ApplyBlockRandomness method and commit beacon fields #1582's first review fixes: the commit field size bound, tolerant block decoding and the kvstore lookup by block hash; d5c374e brings the contract docs, theABCISemVerbump and the split commit size reserve).Open points for review: Tenderdash does not compare a peer's precommit state ID with its own final app hash (not needed for safety: a precommit over another state ID cannot join the honest commit). Like
ExtendVote,ApplyBlockRandomnesshas no deadline on the consensus goroutine, so the application must bound its translation time. The application may be asked to apply the same beacon several times per block (signing, commit checks,FinalizeBlock), so it must answer repeated calls consistently and cheaply.Attribution
Prepared with Claude Code.
🤖 Generated with Claude Code