Skip to content

btcec: reduce allocations - #2559

Open
allocz wants to merge 7 commits into
btcsuite:masterfrom
allocz:btcec_reduce_allocations
Open

allocz wants to merge 7 commits into
btcsuite:masterfrom
allocz:btcec_reduce_allocations

Conversation

@allocz

@allocz allocz commented Jun 30, 2026 •

Copy link
Copy Markdown
Contributor

Change Description

I've optimized public key parsing, signature parsing and signature verification
for both ECDSA and Schnorr, as the benchmarks below show.

Obs: using the following patches for both benchmarks:

golang/go#79708: big.Int.ModInverse allocation zeroing.
decred/dcrd#3714: Public Key parsing and ModNScalar
modulo inverse allocation zeroing.

To avoid forgetting to update the dependencies in the future, even after approval, I think we should wait for upstream releases of both Go and dcrec/secp256k1 containing the patches mentioned above, so that when this code is merged, the benchmark report the exact same results in respect of allocation count.

Merge blockers

  1. go1.28.0
  2. dcrec/secp256k1 v4.4.2

Until the release of the upstream dependencies above, the benchmarks can
be reproduced in a container as described below.

Benchmark improvements:

goos: linux
goarch: amd64
pkg: github.com/btcsuite/btcd/btcec/v2
cpu: Intel(R) Core(TM) i5-10310U CPU @ 1.70GHz
                          │ /data/before │            /data/after             │
                          │    sec/op    │   sec/op     vs base               │
ParseCompressedPubKey-8      28.95µ ± 0%   28.91µ ± 0%  -0.14% (p=0.045 n=11)
ParseUncompressedPubKey-8    578.8n ± 0%   544.2n ± 0%  -5.98% (p=0.000 n=11)
geomean                      4.094µ        3.967µ       -3.10%

                          │ /data/before │              /data/after               │
                          │     B/op     │   B/op     vs base                     │
ParseCompressedPubKey-8       80.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=11)
ParseUncompressedPubKey-8     80.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=11)
geomean                       80.00                   ?                       ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean

                          │ /data/before │               /data/after               │
                          │  allocs/op   │ allocs/op   vs base                     │
ParseCompressedPubKey-8       1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=11)
ParseUncompressedPubKey-8     1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=11)
geomean                       1.000                    ?                       ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean

pkg: github.com/btcsuite/btcd/btcec/v2/ecdsa
                    │ /data/before │             /data/after             │
                    │    sec/op    │   sec/op     vs base                │
SigVerify-8            360.1µ ± 0%   360.3µ ± 0%        ~ (p=0.365 n=11)
ParseDERSignature-8    160.6n ± 0%   129.4n ± 0%  -19.43% (p=0.000 n=11)
ParseBERSignature-8    156.3n ± 0%   126.6n ± 0%  -19.00% (p=0.000 n=11)
geomean                2.083µ        1.807µ       -13.25%

                    │ /data/before │               /data/after               │
                    │     B/op     │    B/op     vs base                     │
SigVerify-8           0.000 ± 0%     0.000 ± 0%         ~ (p=1.000 n=11) ¹
ParseDERSignature-8   64.00 ± 0%      0.00 ± 0%  -100.00% (p=0.000 n=11)
ParseBERSignature-8   64.00 ± 0%      0.00 ± 0%  -100.00% (p=0.000 n=11)
geomean                          ²               ?                       ² ³
¹ all samples are equal
² summaries must be >0 to compute geomean
³ ratios must be >0 to compute geomean

                    │ /data/before │               /data/after               │
                    │  allocs/op   │ allocs/op   vs base                     │
SigVerify-8           0.000 ± 0%     0.000 ± 0%         ~ (p=1.000 n=11) ¹
ParseDERSignature-8   1.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=11)
ParseBERSignature-8   1.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=11)
geomean                          ²               ?                       ² ³
¹ all samples are equal
² summaries must be >0 to compute geomean
³ ratios must be >0 to compute geomean

pkg: github.com/btcsuite/btcd/btcec/v2/schnorr
                 │ /data/before │             /data/after             │
                 │    sec/op    │   sec/op     vs base                │
SigVerify-8         419.3µ ± 0%   418.5µ ± 0%   -0.19% (p=0.002 n=11)
ParsePubKey-8       29.05µ ± 0%   28.96µ ± 0%   -0.30% (p=0.000 n=11)
ParseSignature-8    192.2n ± 0%   147.4n ± 0%  -23.31% (p=0.000 n=11)
geomean             13.28µ        12.13µ        -8.62%

                 │ /data/before │              /data/after               │
                 │     B/op     │   B/op     vs base                     │
SigVerify-8          240.0 ± 0%    0.0 ± 0%  -100.00% (p=0.000 n=11)
ParsePubKey-8        80.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=11)
ParseSignature-8     80.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=11)
ParsePubKey-8        80.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=11)
ParseSignature-8     80.00 ± 0%   0.00 ± 0%  -100.00% (p=0.000 n=11)
geomean              115.4                   ?                       ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean

                 │ /data/before │               /data/after               │
                 │  allocs/op   │ allocs/op   vs base                     │
SigVerify-8          5.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=11)
ParsePubKey-8        1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=11)
ParseSignature-8     1.000 ± 0%   0.000 ± 0%  -100.00% (p=0.000 n=11)
geomean              1.710                    ?                       ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean

The signature parsing time reduced in ~20%. But all the procedures in the
benchmarks above are not doing allocations anymore, reducing the
frequency between garbage collection cycles that must scan all the
reachable heap (that is considerably large in production).

Now we have more CPU available to perform block verification.

Reproducing the benchmarks

Containerfile:

FROM docker.io/library/debian:trixie-slim

ARG DEBIAN_FRONTEND=noninteractive

ENV GOOS=linux
ENV GOARCH=amd64
ENV OLDPATH=${PATH}
ENV PATH=${OLDPATH}:/usr/local/bin/go/bin

RUN apt update && apt install -y git ca-certificates curl bzip2

RUN mkdir /dl && cd /dl \
        && git clone https://github.com/golang/go \
        && git clone https://github.com/btcsuite/btcd

RUN cd /tmp && curl -LO 'https://go.dev/dl/go1.27.3.linux-amd64.tar.gz' \
        && tar -xf go1.27.3.linux-amd64.tar.gz \
        && mv go /usr/local/bin/

# Build Go containing the 79708 patch
RUN cd /dl/go \
        && git checkout 1a3e1568345923aa63e100270ba8af905533ff8f \
        && cd src \
        && ./bootstrap.bash

ENV PATH=${OLDPATH}:/dl/go-linux-amd64-bootstrap/bin:/root/go/bin

# Use dcrec https://github.com/decred/dcrd/pull/3714 and Checkout to PR 2559
# commit `btcec,btcec/ecdsa,btcec/schnorr: add and improve ecc benchmarks`.
RUN cd /dl/btcd \
        && git fetch origin pull/2559/head:2559 \
        && git switch 2559 \
        && git checkout HEAD~6 \
        && go work init \
        && go work use . ./chainhash ./btcec \
        && cd btcec \
        && go get \
                github.com/decred/dcrd/dcrec/secp256k1/v4@82b407dcccb069b7d26e493af44dcc918bb8d686 \
        && go mod tidy \
        && go test . -run=NONE

RUN cd /dl/btcd \
        && go get golang.org/x/perf/cmd/benchstat \
        && go install golang.org/x/perf/cmd/benchstat

COPY init.sh /init.sh

RUN chmod +x /init.sh

ENTRYPOINT ["/init.sh"]

init.sh:

#!/usr/bin/env bash

set -e

cd /dl/btcd/btcec

bench='Benchmark(ParseCompressedPubKey|ParseUncompressedPubKey|ParseDERSignature|ParseBERSignature|ParsePubKey|SigVerify|ParseSignature)'

go test . ./ecdsa ./schnorr \
        -bench=${bench} \
        -benchmem -count=11 -run=NONE | tee /data/before

git stash push
git switch 2559
git stash pop

go test . ./ecdsa ./schnorr \
        -bench=${bench} \
        -benchmem -count=11 -run=NONE | tee /data/after

benchstat /data/before /data/after | tee /data/stats

Build the image:

podman build \
        -f Containerfile \
        --tag="btcd_pr_2559" \
        .

Start the container that will run the benchmarks:

mkdir -p data

podman run --rm \
        -v=data:/data \
        --name="btcd_pr_2559" \
        "btcd_pr_2559"

Pull Request Checklist

Testing

  • Your PR passes all CI checks.
  • Tests covering the positive and negative (error paths) are included.
  • Bug fixes contain tests triggering the bug to prevent regressions.

Code Style and Documentation

📝 Please see our Contribution Guidelines for further guidance.

@allocz

allocz commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

CC @kcalvinalvin

@Roasbeef

Copy link
Copy Markdown
Member

Approved CI run.

Comment thread btcec/pubkey.go Resolved
Comment thread btcec/schnorr/pubkey.go Resolved
Comment thread btcec/ecdsa/signature.go Resolved
@allocz
allocz force-pushed the btcec_reduce_allocations branch 2 times, most recently from 1f3d515 to ac92fde Compare July 4, 2026 12:44
@allocz

allocz commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

The signature parsing wins are real with today's dependencies. ecdsa SigVerify stays at 12 allocs on go1.27.0 (10 with dcrd#3714), so the 0 in the description still depends on golang/go#79708.

Yep, this PR is blocked until we get new releases of the Go compiler and dcrec/secp256k1 containing the patches.

@Roasbeef Roasbeef left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No actionable correctness regression found in the reviewed parsing, serialization, hashing, and verification changes. go test ./... passed from btcec at this head. I haven't independently reproduced the allocation/performance benchmarks.

Holding off on approval until this is rebased to resolve the conflicts and the Go/secp upstream-release prerequisites mentioned in the PR description are checked. Please confirm the dependency/toolchain versions containing those optimizations so the reported allocation results can be reproduced.

@allocz

allocz commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

@Roasbeef, I've updated the description with the instructions to reproduce the benchmarks. Also, specified that we need to wait until go1.28.0 and dcrec v4.4.2 releases to merge this PR collecting 100% of the performance/allocation improvements.

To optimize critical procedures used in validation loop, we first need
all of them covered by benchmarks, so that we can have a baseline for
future optimizations.

btcec added benchmarks:
* BenchmarkParseUncompressedPubkey

btcec/ecdsa added benchmarks:
* BenchmarkParseDERSignature
* BenchmarkParseBERSignature

btcec/schnorr added benchmarks:
* BenchmarkParsePubKey
* BenchmarkParseSignature

btcec/ecdsa benchmark improvements:
* Avoid unneccessary allocations on BenchmarkSigVerify.
* Avoid unneccessary allocations on BenchmarkParseCompressedPubKey.
Previously, changing DER boolean argument of parseSig was breaking
VerifyLowS behavior without breaking the tests. New sentinel errors and
entries to the VerifyLowS test table were added, effectively catching
regressions in the implementation.
Added unexported function that does not return a pointer to the public
key to execute the key parsing, letting the exported ParsePubKey short
and simple, making it eligible to inlining and therefore zeroing out
allocations.
Added unexported function that does not return a pointer to the public
key to execute the key parsing, letting the exported ParsePubKey short
and simple, making it eligible to inlining and therefore zeroing out
allocations.
btcec/schnorr: replace usage of SerializePubKey for serializePubKey32,
which writes to a byte slice instead of allocating and returning a new
one.

chainhash: keep TaggedHash short and simple to make it eligible to
inlining and avoid allocations.
Make ParseSignature and ParseDERSignature short and simple, making the
procedures eligible to inlining and therefore avoiding allocations.
Make ParseSignature short and simple, becoming eligible to inlining
which avoids allocations.
@allocz
allocz force-pushed the btcec_reduce_allocations branch from ac92fde to 4182225 Compare November 9, 2026 18:40
@allocz

allocz commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

Rebased on top of master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants