Skip to content

Chore/bench weights spec 17 - #156

Merged
nol4lej merged 4 commits into
mainfrom
chore/bench-weights-spec-17
Oct 6, 2026
Merged

nol4lej merged 4 commits into
mainfrom
chore/bench-weights-spec-17

Conversation

@nol4lej

@nol4lej nol4lej commented Oct 6, 2026

Copy link
Copy Markdown
Member

Summary

Regenerates every versioned weight on the reference hardware (Hetzner CPX62, ubuntu-32gb-fsn1-1, 50/20) after spec 17, and fixes the benchmark runner. Before this fix, the runner could not complete on the CPX62.

Runner (scripts/benchmarks/run_benchmarks.sh)

  • --no-median-slopes. In the 2606 CLI, the median-slopes analysis grows with repeat on any benchmark that has a component. It is only printed: weights come from min-squares, the default of --output-analysis. Even so, it pushed pallet_relayer and pallet_ismp_messaging past 31 GB, and they were OOM-killed on the CPX62. sha3fips was OOM-killed even on a 64 GB host.

    Peak memory with the flag, measured locally at 50/20:

    Benchmark Peak RSS Time
    relayer 0.43 GB 98 s
    ismp 2.1 GB 287 s
    sha3fips 2.3 GB 191 s
    set_allowed_selectors (5×20) 0.40 GB, was 3.8 GB 13 s, was 131 s

    Ruled out first: PoV mode, --no-storage-info, --no-verify, --disable-proof-recording, and T::X::get() ranges.

  • Runtime-side weights are rewritten to the upstream trait. Applies to every file under template/runtime/src/weights/, today ismp_grandpa. The script drops the local trait WeightInfo and the impl for (), and points the impl at <pallet>::weights::WeightInfo.

    • Before, this was a manual step. Forgetting it broke the runtime build, and every later run of the script with it, since the script rebuilds first.
    • The run fails if the rewrite does not apply.
  • Proof-size guard. After regenerating pallet_ismp_messaging, the script runs declared_proof_sizes_stay_within_a_sane_ceiling. It fails the run when the CLI writes a corrupted proof_size (polkadot-sdk#13066).

  • Comments and the exit-137 message are updated: they still said "~15 GB" and "no flag makes that fit".

Weights (CPX62, 2026-10-06)

  • Regenerated: shielded-pool, zk-verifier, relayer, validator-set, ismp-messaging, ismp_grandpa.

  • relayer and ismp had not been regenerated since August and September, before the SDK 2606 migration.

  • shield / shield_batch:

    Extrinsic Weight PoV
    shield 822 M ps 4707
    shield_batch, per entry 785 M ps 2475

    This is the pallet's own cost, mostly the Poseidon Merkle insert. Groth16 verification is not included: it is added separately through verification_weight (~11.7 G ps).

  • Hand-corrected dispatch_get proof_size. The CLI wrote 8433417681689641984 against 283 measured bytes. The base is now 3748, the value --steps 3 and --steps 5 produce. Proof size is measured storage and does not depend on the host, and 3748 also equals dispatch_post's base on this run, since both walk RequestCommitments. The per-key slope (k × 1874) and every execution time are the CPX62's. The "Hand-corrected value" header is restored for dispatch_get.

Test plan

  • Full run on the CPX62: [3/3] All 12 benchmark(s) completed successfully.
  • cargo check -p orbinum-runtime
  • cargo test: ismp-messaging 86, shielded-pool 390, zk-verifier 128, relayer 106, validator-set 36.
  • No other proof_size of 8 digits or more in the 6 regenerated files.
  • Runner locally at 50/20: relayer, the ismp group and sha3fips complete. The rewritten ismp_grandpa.rs compiles, and it is byte-identical to the hand-fixed one on a simulated generator output.
  • Proof-size guard: passes on the committed weights, fails on an injected 8433417681689641984, and is skipped for other pallets.

@nol4lej
nol4lej merged commit 9300d78 into main Oct 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant