Skip to content

pool: payout_modes = "pool_only", sv2_coinbase_output_info and Block Found attribution (mara-0.9 2/3) - #35

Open
average-gary wants to merge 10 commits into
mara-0.9from
m09/pool
Open

average-gary wants to merge 10 commits into
mara-0.9from
m09/pool

Conversation

@average-gary

Copy link
Copy Markdown
Collaborator

pool: payout_modes = "pool_only", sv2_coinbase_output_info and Block Found attribution

The second PR of the mara-0.9 port, stacked on the base PR (#34). It ports #24, #23 and the pool Block Found attribution (8ad7f71a) onto v0.9.2. Each patch was rewritten from its essence (behaviour, invariants, keys, metrics, log lines, tests), not rebased. The 0.8 commits are 0ad07d30, 67a2827b and 8ad7f71a.

1. payout_modes (#24)

2. One coinbase decoder

  • build_declared_coinbase_tx is the only reconstruction of a declared coinbase. It's used by the pool_only check, declaration validation, the SetCustomMiningJob output comparison and PushSolution block building. Upstream's get_coinbase_tx now calls it.

    • A first version of this port left upstream's second decoder in place. The two disagreed on non-segwit splits, so a crafted declaration could pass pool_only under one and pay a miner under the other.
  • Under pool_only, non-segwit declarations are also refused. Only a non-Core TP without a witness commitment would produce one, and the cost is a refused declaration, not a payout leak.

  • Under Any this is stricter than upstream:

    • trailing bytes, more than one input, or a scriptSig over 100 bytes are refused;
    • short prefixes are an error instead of a panic, and the usize underflow in upstream's version is gone;
    • non-segwit prefixes now decode correctly (offset 41).

    0.9.2's own JDC never produces these, and all 22 jd_integration ITs pass. A third-party JDC that pads its suffix would now be refused. Worth proposing upstream.

3. deny_unknown_fields

  • It's on PoolConfig, [jds] and TemplateProviderType, which JDC configs share. An unknown key, or a typo such as payout_mode, is a startup error, never a silently ignored payout setting.
  • All pool and JDC config examples load (new tests). pool-v4-infra's rendered dev, staging and prod pool.toml were checked to load on this build.
  • Side effect: a key that a future upstream release adds is a hard failure until pool-v4's TOML and the fork agree.

4. Monitoring (#23)

  • The pool becomes a server source: /api/v1/server returns 200 with zero channels.
  • New metrics, with names, labels and help strings identical to mara:
    • sv2_coinbase_output_info{script_hex,address}
    • sv2_monitoring_snapshot_age_seconds
  • They're built from state, so stratum-core 0.7.0's #2411 (group→standard jobs no longer duplicate outputs) doesn't change them. Worth a sanity check on dev anyway.
  • JDC reports every live output set, a judgment call compared with 0.8's last token only. pool-v4 doesn't run JDC.

5. Block Found

  • The SubmitShares{Standard,Extended}: 💰 Block Found!!! 💰{hash} text is unchanged, with the attribution fields added (downstream, channel, user identity, worker). The block poster's HASH_RE still matches.

Public API (stratum-apps and others)

  • New:
    • ServerInfo.coinbase_outputs (a new required field)
    • CoinbaseOutputInfo
    • AllowedPayoutModes
    • build_declared_coinbase_tx, decode_declared_coinbase_tx, decode_coinbase_tx_parts, validate_pool_only_outputs
    • PayoutModeError::NotAllowed
    • PayoutValidationError::NonPoolOutput
    • TemplateProviderType::network()
    • JobDeclarator::with_payout_modes

Tests

Local only.

  • Every commit: fmt, clippy -D warnings for pool, jd-server, jd-client, translator and stratum-apps --all-features, plus their unit tests.
  • On the stack: --locked release build, cargo audit with 0 vulnerabilities, and pool_payout_modes plus monitoring_integration 16/16.
  • Mutations:
    • Removing the pool_only check from either open handler fails the payout ITs.
    • Removing the JDS check fails pool_only_jds_refuses_declared_coinbases_that_pay_anyone_else.
    • Restoring upstream's old get_coinbase_tx fails the agreement tests, which panic on the underflow.
  • Reviews: an Opus adversarial review (one blocker, the two decoders, fixed and re-approved) and a Fable final review (approve).

Merge method

Please rebase-merge after the base PR, not squash.

@gimballock gimballock left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial pass at 245aa39d. No must-fix: pool_only holds on every path we could find.

The gate

  • Both open handlers check before payout_is_compatible and before extranonce allocation (mining_message_handler.rs:177, :440). Nothing else calls PayoutMode::try_from.
  • UpdateChannel doesn't touch payout, and group and template jobs use the stored mode.
  • SCMJ needs a token only a checked declaration can activate.
  • The embedded JDS requires every output to pay coinbase_reward_script exactly, or to be a zero-value OP_RETURN.
  • One decoder (build_declared_coinbase_tx) serves declaration, SCMJ and PushSolution, so there's no gap between the coinbase that's checked and the one that's mined.
  • The hardened decoder refuses non-minimal varints, a scriptSig over 100 bytes, input counts other than one, and trailing bytes.
  • sv2_coinbase_output_info and the Block Found lines are identical to 0.8.

pool-v4-infra compatibility with deny_unknown_fields

  • All six cell configs pool-v4-infra renders today (dev, staging and prod, from main 592d790) load with this PR's pool_sv2. Nothing is rejected or needs renaming.
  • The chart sets no POOL__* variables. The pod's single-underscore POOL_* and service-link variables can't reach config-rs's pool__ prefix.
  • From now on the chart and the image have to change keys together. Two chart comments (templates/pool-cell-config.yaml:23, :143) still say sv2-apps never denies unknown fields. We'll fix those on our side.

Should-fix (fine as a follow-up)

  • The SCMJ output comparison is untested (jd-server/.../bitcoin_core_ipc.rs:1004). It's the only thing stopping a custom job from paying out differently from its checked declaration. Disabling it (if false && …) passes jd-server lib, pool_payout_modes and jd_integration, and no test references its error code.
    • Failure scenario: a refactor loosens it, a JDC declares a pool-only coinbase, then sends an SCMJ paying itself, and the shares are credited.
    • This was "open, not done" on #24 too.

Nits

  • The exact-script match is unguarded by tests. Turning payout.rs:58's comparison into a prefix match survives every test. Add a case with a superset or prefix of the pool script.
  • get_coinbase_tx_agrees_with_the_pool_only_decoder now compares the decoder with itself. All it does is stop a second decoder from coming back.
  • Underpaying the pool is still accepted (stratum-mining#395, already known). Under FPPS it lets a JDC withhold block value while still being credited.

Run

  • pool_sv2 11/11, jd_server_sv2 15/15, stratum-apps lib 164.
  • clippy clean.
  • 12 mutations of the gate, the decoder and the JDS check. 10 were caught; the two above survived.

average-gary and others added 4 commits November 16, 2026 17:03
The port lands as PRs stacked on each other, each targeting the
previous one's `m09/*` branch rather than `mara-0.9`. With the PR
checks triggered only by PRs into `mara-0.9`, only the bottom PR
of the stack got CI, and none of the others did. Also triggering
on PRs into `m09/**` runs fmt, clippy, the tests, the integration
tests, lockfiles, semver and MSRV on every PR of the stack. The
push triggers stay on `mara-0.9` only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both pool Block Found log lines now carry structured tracing fields:
downstream_id, channel_id, the channel's user_identity and template_id.
Extended channels also log the per-share worker-hashrate-tracking TLV
identity, which was parsed and then discarded. The placeholder if-let
that ignored it is removed.

The block hash stays first in the message, so existing `Block Found`
greps and dashboards keep matching. The fields feed the block-found
archive, which needs to know who found each block.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Each app can now report the scripts its own coinbase outputs pay, so an
operator can alert on what an app actually puts in its jobs rather than
on what its config file says it should.

- ServerInfo gains `coinbase_outputs: Vec<CoinbaseOutputInfo>`. The
  snapshot cache exports it as
  `sv2_coinbase_output_info{script_hex,address} 1`, refreshed with the
  rest of the snapshot so a scrape never takes business locks. Stale
  label sets are removed like the other per-label server gauges. The
  gauge exists only when server metrics are enabled.
- Addresses are rendered with `Address::from_script` for the network
  named by the new `TemplateProviderType::network()`. With `Sv2Tp` the
  network is unknown and the address is empty, as it is for scripts
  with no address form.
- `sv2_monitoring_snapshot_age_seconds` is a system metric set at
  scrape time from the snapshot timestamp. Every other server metric
  keeps serving its last value when the refresh stops, so without it a
  frozen snapshot looks healthy.

ServerInfo gained a required field, so the translator reports no
outputs and the JD client reports an empty list for now; the producers
follow in the next commit.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Fill the `coinbase_outputs` that ServerInfo now carries, so
`sv2_coinbase_output_info` shows what each app really pays.

Pool: implement ServerMonitoring on the channel manager with zero
channels. It reports the pool's loaded coinbase outputs plus the
outputs of each connected downstream's payout mode, which are only set
after a successful channel open. Downstreams that require custom work
are skipped, since the pool builds no jobs for them. The scripts are
read from state rather than from the jobs sent, so outputs inside
JD-declared custom jobs are not reported, and a client whose channels
use different payout modes reports only its latest.

Because the pool is now a server source, /api/v1/server answers 200
with zero channels instead of 404, and the zero-valued sv2_server_*
gauges are exported. The monitoring integration tests are updated.

JD client: report the scripts of every live set of coinbase outputs,
not only the current one. Jobs built from an older set stay minable
until the chain tip moves, so an output they pay can still win a
block. The set is read before the upstream channel lock is taken.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@average-gary
average-gary changed the base branch from m09/base to mara-0.9 November 16, 2026 22:03
@average-gary

Copy link
Copy Markdown
Collaborator Author

Thanks, @gimballock.

Restack: #34 was rebase-merged into mara-0.9 (2ce137be), so this PR now targets mara-0.9 directly. It's rebased onto it, and the content is unchanged from 245aa39d, which you reviewed. One commit is added at the bottom: ci: run the PR checks on PRs stacked on the mara-0.9 port. It adds m09/** to the pull_request branches, so #36, which targets this branch, gets checks too.

CI: CI still can't run. GitHub Actions is disabled for marafoundation/sv2-apps. A workflow_dispatch returns HTTP 422: Actions has been disabled for this repository, and no run has been created since 2026-10-06. That's the "dead CI" since then; the triggers aren't the cause. I've asked for it to be re-enabled. Once it is, the checks run here and on #36 on the next push.

Should-fix (SCMJ output comparison): a follow-up commit is in progress. It adds a test where a pool-only declaration is followed by an SCMJ that pays out differently, which must be refused, and one with matching outputs, which must be accepted. It is mutation-checked against if false && ….

Nits:

  • Exact-script match: a test for a strict prefix and a superset of the pool script, both refused, mutation-checked against starts_with.
  • The agreement test: you're right that it now only stops a second decoder from coming back. I'll say so in its doc comment rather than pretend it compares two implementations.
  • Underpaying the pool (Truncate user_identity to 32 bytes. stratum-mining/sv2-apps#395): out of scope here, tracked there.
  • The chart comments at pool-cell-config.yaml:23,:143: noted that you'll fix those on the infra side.

average-gary and others added 5 commits November 16, 2026 17:10
A pool that only pays itself needs two checks the payout module could
not express: which payout modes a user_identity may select, and whether
a coinbase declared by a JD client pays anyone but the pool.

- AllowedPayoutModes {Any, PoolOnly}: `check` chains after
  PayoutMode::try_from and refuses everything but FullDonation under
  PoolOnly with PayoutModeError::NotAllowed. Any changes nothing.
- validate_pool_only_outputs: every output pays the pool script or is
  a zero-value OP_RETURN, and at least one pays the pool. The total is
  not compared with the block reward; value left unclaimed pays nobody.
- build_declared_coinbase_tx: the one reconstruction of a coinbase
  from the split in DeclareMiningJob, `prefix | extranonce | suffix`,
  where the extranonce is zeros or a given value of the exact size the
  prefix implies. decode_declared_coinbase_tx is it with zeros. The
  split does not carry the extranonce size, so
  declared_coinbase_extranonce_size reads it from the scriptSig length
  in the prefix: at byte 43 behind a segwit marker and flag, else at
  byte 41. Any consumer that rebuilds a declared coinbase must use
  this, so the transaction a payout check approves is the transaction
  that is validated, compared and mined; two readers of the same
  bytes could shift the suffix differently and disagree on who is
  paid.
- A prefix is refused, with no panic and no allocation sized by a
  client-declared length, if it is too short, does not hold exactly
  one input, ends past the scriptSig (checked arithmetic), or declares
  a scriptSig over the 100 byte consensus limit. Bytes left after the
  transaction are refused too.
- decode_coinbase_tx_parts is the shared reconstruction for splits
  that carry the extranonce size, so the existing payout verification
  and the declared-coinbase decoder cannot diverge.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The pool honours whatever payout a user_identity encodes:
`sri/solo/<addr>` or a bare address sends the whole reward to <addr>,
and `sri/donate/<pct>/<addr>` sends part of it. The embedded JDS
accepts any declared coinbase, whoever it pays. A pool that pays only
its operator has no way to turn either off.

A new pool config key, `payout_modes`, is "any" by default, which
changes nothing. With "pool_only":

- Opening a standard or extended channel whose identity selects solo,
  legacy solo or donate is refused with `invalid-user-identity`.
  Identities with no payout mode and `sri/donate[/<worker>]` still pay
  the pool.
  The check chains after PayoutMode::try_from at both open handlers,
  before the per-connection payout compatibility check and before
  extranonce allocation, so a refused identity neither binds the
  connection's payout policy nor takes an extranonce prefix.
- The embedded JDS refuses a DeclareMiningJob with `invalid-coinbase-tx`
  unless its coinbase pays coinbase_reward_script and nothing else
  apart from zero-value OP_RETURN outputs. This runs after the token
  checks and before the validation backend. A declaration answered
  with ProvideMissingTransactions is re-validated from the stored
  original message, so this first check covers it too. Only coinbases
  serialised as segwit (marker and flag after the version, which every
  JD client declares) are accepted; any other prefix is refused with
  the same code before it is decoded.

A declared coinbase is rebuilt in one place, stratum-apps
build_declared_coinbase_tx. The payout check, the validation of the
declaration, the comparison with a SetCustomMiningJob's outputs and
the block built from a PushSolution (get_coinbase_tx) all call it, so
the transaction the pool_only check approves is the one that is
validated and mined. Malformed prefixes are an error, never a panic.

The pool owns the setting: the standalone jd_server binary has no
payout_modes key and always runs `Any`. Only the `[jds]` embedded in a
pool inherits the pool's payout_modes.

The refusal log now prints the error, which says which payout mode was
refused, instead of one fixed text for every failure. jd-server enables
the stratum-apps `payout` feature, and every pool example config
documents the key.

The integration tests start their own pools rather than changing the
shared helpers. Besides one connection per identity, one test opens a
pool identity and then a miner-paying one on the same connection: under
pool_only the second is refused as an invalid identity, under any as an
incompatible payout mode.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A misspelled key was silently dropped, so `payout_mode = "pool_only"`
(singular) left payout_modes at the permissive "any" and the pool
started paying miners. PoolConfig, its [jds] table (JDSPartialConfig)
and every TemplateProviderType variant now reject unknown keys, from
the TOML file and from POOL__* environment overrides alike.

TemplateProviderType is shared with the JD client, which gets the same
behaviour: a JDC config with a stray key under
[template_provider_type.*] no longer starts. All pool and JD client
example configs still load.

Tests cover an unknown key at the top level, in [jds] and in a template
provider table, an unknown environment override, and that every pool
and every JD client example config loads.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The comparison of a SetCustomMiningJob's coinbase outputs with the
declared ones is the only check that ties the job a JDC mines to the
coinbase that was checked at declaration time, including the one
payout_modes = "pool_only" accepts. Nothing tested it: replacing it
with `if false && ...` passed every test.

Drive handle_set_custom_mining_job directly. A job that pays out as
declared is accepted. A job whose outputs pay another script, pay a
different amount, add an output or have none is refused with
ERROR_CODE_SET_CUSTOM_MINING_JOB_INVALID_COINBASE_TX_OUTPUTS.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
`cargo audit --deny warnings`, as pool-v4-infra's sv2 rust gate runs
it, failed on the root lockfile on two counts.

- RUSTSEC-2026-0538: quinn-udp 0.5.14's receive loop can spin after a
  truncated datagram. It is patched in 0.5.16. No shipped binary
  links quinn, but the gate audits the whole lockfile.
- Four unmaintained or yanked crates (async-std, instant, pin-utils,
  async-executor), all pulled in by criterion's `stable` feature,
  which enables every async benchmark runtime. The benches use none
  of them, so criterion is built without it.

That leaves RUSTSEC-2026-0543 (chrono, informational "unmaintained",
through asic-rs), which needs a waiver in the gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@based64god based64god left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think this looks good. Sorry it took a while. Fairly difficult to review a diff this large.

@average-gary

Copy link
Copy Markdown
Collaborator Author

Follow-ups pushed (head 5e82f360). The six commits you reviewed are unchanged, apart from the two fixups folded in below.

  • 0c5595c0 test(jd-server): refuse a custom job that pays out differently. This is your should-fix. It drives handle_set_custom_mining_job with a job that pays out as declared (accepted), then with another script, another amount, an extra output and no outputs (all refused with …INVALID_COINBASE_TX_OUTPUTS). if false && … on the comparison now fails it.
  • The exact-script nit is folded into dcebf387: a strict prefix and a superset of the pool script are both NonPoolOutput, and a starts_with either way fails. The agreement-test doc nit is folded into 944d74d9.
  • 5e82f360 fix(deps): bump quinn-udp to 0.5.16 and drop criterion's async runtimes. This is new. pool-v4's gate runs cargo audit --deny warnings, which flagged RUSTSEC-2026-0538 and four unmaintained or yanked crates pulled in by criterion's stable feature. The benches use none of them. Only RUSTSEC-2026-0543 (chrono, via asic-rs) is left, and the gate waives it. No shipped binary's dependency tree changes.

Reviews: Opus adversarial (approve) and Fable final (approve). CI is still blocked on the fork's Actions banner; I'll ask for approval once the checks are green.

The first CI run on the port failed twice for reasons unrelated to
the code.

- `bnjbvr/cargo-machete@main` builds cargo-machete from source with
  the workspace's pinned toolchain. Its newest release depends on
  cargo-platform 0.3.3, which needs rustc 1.91, so the install
  failed on 1.88. Install the released binary instead.
- `cargo semver-checks` compares each crate with its last published
  version on crates.io. The fork never publishes crates and keeps
  upstream's versions, so the API additions the Mara patches make
  (new public fields and enum variants, listed in each PR) always
  read as major-version breaks. Run the check only on
  stratum-mining/sv2-apps, as release-apps already is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@average-gary

Copy link
Copy Markdown
Collaborator Author

Ready for approval: @gimballock @based64god (head dd20ecf5, rebase-merge into mara-0.9)

CI is the first run on the fork since 2026-10-06; Actions is re-enabled. It's green on Linux, which is the platform that ships:

  • Format, Clippy and Unused Dependencies (cargo machete)
  • Test Suite (ubuntu-latest)
  • Integration Tests (ubuntu-latest)
  • Lockfiles, MSRV 1.88, Validate OpenAPI

Two things didn't run:

  • The macOS jobs (Test Suite and Integration Tests on macos-latest) were never given a runner: "not acquired … capacity constraints, macOS arm64". They didn't fail. A retry is running and I'll note the result here. Every local run of this stack (unit, integration, mutations, rig) was on macOS aarch64.
  • Semver check is skipped on the fork by design (dd20ecf5). The fork never publishes crates, so the documented API additions would always read as major breaks.

Since your review (245aa39d):

  • 0c5595c0: SCMJ output-comparison test (your should-fix).
  • The exact-script and agreement-test nits, folded into dcebf387 and 944d74d9.
  • 5e82f360: quinn-udp 0.5.16 and criterion without stable, so cargo audit --deny warnings leaves only the waived chrono notice.
  • edc56c18 / dd20ecf5: CI fixes, m09/** triggers, released cargo-machete, semver skipped on the fork.

Reviews: Opus adversarial and Fable final on every change.

git range-diff 245aa39d~6..245aa39d origin/mara-0.9..dd20ecf5 shows the deltas.

@gimballock gimballock left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving dd20ecf5. Re-reviewed the deltas since 245aa39d:

  • git range-diff: the reviewed commits are unchanged except the two that fold in my nits (exact-script prefix and superset cases; the agreement-test doc comment).
  • 5e82f360: the only version change in Cargo.lock is quinn-udp 0.5.14 → 0.5.16; everything else is removals from dropping criterion's stable. cargo audit now reports only the chrono "unmaintained" notice, which pool-v4-infra waives.
  • 0c5595c0: replacing the SetCustomMiningJob output comparison with if false && … fails custom_job_paying_out_differently_from_its_declaration_is_refused, so the should-fix is pinned.
  • jd-server lib: 17/17.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants