Repository navigation
dev: add [development.force_mismatch] to force a bank hash mismatch - #12474
Open
nishk-jump wants to merge 1 commit into
Open
nishk-jump wants to merge 1 commit into
nishk-jump wants to merge 1 commit into
Conversation
Adds a firedancer-dev development knob to deliberately force a bank hash
mismatch, for exercising the mismatch-handling paths.
[development.force_mismatch]
enabled = false
at_slot = 0 # force on a specific slot
after = 0 # force after this many slots; both unset => first slot
One config, two consumers, each with its own reaction:
- backtest: the backt tile forces its ledger comparison to mismatch,
which logs the mismatch and exits (as a real mismatch does).
- live/network: the replay tile forces the Alpenglow footer comparison
to mismatch once (latched), marking the block dead and continuing.
In a backtest run both tiles exist, so the backt tile owns the knob and
replay's copy is disarmed in the backtest command topology. at_slot and
after are mutually exclusive and rejected together at config-parse time.
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Live consensus fault injection and subsequent recovery need targeted runtime validation and human review.
1 open issue
What changed in this PR
Adds configurable bank-hash mismatch injection to exercise Firedancer’s failure-handling paths.
Changes:
- Adds configuration, validation, and JSON output for mismatch triggers.
- Supports one-shot Alpenglow replay injection and backtest comparison failures.
- Keeps backtest injection separate from replay injection.
| File | Description |
|---|---|
| src/discof/replay/fd_replay_tile.c | Implements one-shot replay injection. |
| src/discof/replay/fd_replay_tile_private.h | Stores trigger and latch state. |
| src/discof/backtest/fd_backtest_tile.c | Forces backtest comparison failures. |
| src/disco/topo/fd_topo.h | Adds tile configuration fields. |
| src/app/shared/fd_config.h | Defines mismatch settings. |
| src/app/shared/fd_config_parse.c | Parses settings and rejects conflicting triggers. |
| src/app/shared/fd_config_json.c | Serializes mismatch settings. |
| src/app/firedancer/topology.c | Passes settings to tiles. |
| src/app/firedancer/config/default.toml | Documents defaults and trigger behavior. |
| src/app/firedancer-dev/commands/backtest.c | Disables replay injection during backtests. |
Review effort: Balanced
Comment on lines
+1961
to
+1964
| int force = ctx->force_mismatch && !ctx->force_mismatch_fired && | ||
| ( ( ctx->force_mismatch_at_slot && bank->f.slot==ctx->force_mismatch_at_slot ) || | ||
| ( ctx->force_mismatch_after && ctx->blocks_completed==ctx->force_mismatch_after ) || | ||
| ( !ctx->force_mismatch_at_slot && !ctx->force_mismatch_after && !ctx->blocks_completed ) ); |
┌─ ⚡ PERF · 61d5129 vs main@ccf8d76 ─────────────────────────────────
│ SUITE BASELINE NEW Δ
│ replay tps, mainnet 30,278 tps 30,334 tps · +0.18%
│ bench tps, localnet 668,399 tps 663,560 tps · -0.72%
│ snapshot load, testnet 3.87 s 3.86 s · -0.05%
│ mem total, mainnet 146.36 GiB 146.36 GiB · 0.00%
│ mem total, testnet 87.25 GiB 87.25 GiB · 0.00%
│ mem total, ag mainnet 147.17 GiB 147.17 GiB · 0.00%
│ clean compile, firedancer 3.43 s 3.46 s · +0.80%
│ binary size, firedancer 59.94 MB 59.96 MB · +0.03%
├─────────────────────────────────────────────────────────────────────
@@ 0 REGRESSIONS · 0 WARNINGS · 0 IMPROVED · 8 NOISE @@
└───────────────────────────────────────────────────────────────────── |
mmcgee-jump
reviewed
Oct 21, 2026
| exercise the dead-block path. ctx->blocks_completed is the count | ||
| of blocks that completed this check before this one, so "first | ||
| block" is blocks_completed==0 and "after N" is blocks_completed==N. */ | ||
| int force = ctx->force_mismatch && !ctx->force_mismatch_fired && |
Contributor
There was a problem hiding this comment.
Really don't like having this in the production binary, is there another way to do this?
Contributor
There was a problem hiding this comment.
I would try keep the logic in the backtest tile, can you alter the shred or change the bank hash or something?
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.