Skip to content

build: optimise libsecp256k1 in dev/test builds (BIP-352 vector tests 247s -> 28s on CI) - #25

Merged
setavenger merged 1 commit into
integrationfrom
secp256k1-dev-opt
Oct 1, 2026
Merged

setavenger merged 1 commit into
integrationfrom
secp256k1-dev-opt

Conversation

@setavenger

@setavenger setavenger commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

Problem

secp256k1-sys compiles the C libsecp256k1 at the profile's opt-level, so every debug and test build runs EC operations at -O0. The BIP-352 vector tests in blindbit-lib feel this most. The K_max vector scans 2324 outputs quadratically, and the blindbit-lib test binary spends about 3 minutes of every CI run on it: 181 s on integration, and 247 s once #18 un-ignores the live-path K_max test.

Change

The root Cargo.toml gets [profile.dev.package.secp256k1-sys] opt-level = 3, and the test profile inherits it. That one C library is the only thing optimised; the rest of the dev build is unchanged. Debug builds of the daemon also scan faster.

I also measured optimising the secp256k1 and bdk_sp Rust crates. It saved less than 5% more CPU time on the K_max test locally (18.9 s → 18.1 s), so I left them alone. This keeps every vector test in the normal test job. The alternative of a separate job or a release-mode run would add a second full compile per PR.

Timings

CI (ubuntu-24.04; no rust-cache hit in any of these runs):

run blindbit-lib test binary Test workspace step whole job
integration code (#16, run) 181.0 s 355 s 8m28s
#18 without this change, K_max live test on (run) 246.9 s 420 s 9m45s
#18 with this change (run) 27.8 s 145 s 4m49s
this PR (572768d on integration, run) 30.5 s 200 s 6m13s

The test-profile compile without a cache took 2m48s on this PR and 2m52s on #16, so the override costs no measurable compile time. Runner hardware does vary: the same 12 tests once took 94.6 s (run). Even against that run this PR is 3x faster, and 6–8x faster against the others.

CI is green on 572768d: blindbit-lib reports 12 passed; 0 failed; 0 ignored, and the clippy step passes.

Locally (#18's head, a private target dir, a loaded 6-core machine), the live K_max test drops from 99.4 s to 18.9 s of CPU time, 5.3x faster. Both builds used --config to set the opt-level explicitly. The commit message's figures (219 s → 41 s) came from a target dir shared with other checkouts, so these replace them; the ratio is about the same.

Motivated by SNB-540 (the #18 CI-cost question). This is independent of #17–#22 and can land in any order.

secp256k1-sys builds the C libsecp256k1 with the profile's opt-level, so
debug and test builds run every EC operation at -O0. The BIP-352 vector
tests feel this most: the K_max vector scans 2324 outputs quadratically,
and the blindbit-lib test binary spent ~3 minutes of every CI run on it
(~4 minutes once SNB-540 un-ignores the live-path K_max test).

Set `opt-level = 3` for secp256k1-sys in the dev profile (the test profile
inherits it). Only that one C library changes; the rest of the dev build is
untouched. Locally the live K_max test drops from 219s to 41s of CPU time;
also optimising the secp256k1 and bdk_sp Rust crates gained <5% more, so
they stay at the default. Debug builds of the daemon scan faster too.

SNB-540

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
setavenger added a commit that referenced this pull request Oct 1, 2026
The secp256k1-sys `opt-level = 3` override speeds up every PR's test run,
not just this one, so it now lands separately as #25
against integration. Revert it here, with the one test comment that
referred to it, so this PR only carries the K_max change. Until #25 is in,
this branch's CI pays the ~66s the un-ignored K_max test costs at -O0.

SNB-540

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
setavenger added a commit that referenced this pull request Oct 1, 2026
The T0 harness replayed each block through a hand copy of the scan loop's
per-block steps, and that copy had already fallen behind: the loop on the
fix stack records owned outputs (sync_owned_outputs) after
apply_matched_block and the copy did not, so nine spends were carried as
KNOWN_SPEND_GAP.

process_block now drives Scanner::scan_block_stream, the per-block loop
scan_block_range and watch_chain run, with the oracle's message streamed
in-process and the full block served in place of the P2P fetch (the
stream-safety test hooks from #20/#21). There is no copy left to drift: a
step added to the loop is exercised here. State files the loop now saves
are removed when a scenario ends.

KNOWN_SPEND_GAP and its assertions are gone; all nine scenarios must pass
with their spends. Dropping the sync_owned_outputs() call from the loop
turns exactly those nine spends red again (k-gap, labelled-multi,
spent-outputs, dust-boundary, rescan).

Fixture provenance now names testkit 68f9f1c, the merge of testkit PR #25
into master; the fixtures are byte-identical to the export at b1f962e.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@setavenger
setavenger merged commit c51b8ed into integration Oct 1, 2026
1 check passed
@setavenger
setavenger deleted the secp256k1-dev-opt branch October 1, 2026 20:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant