Skip to content

Allocate the TLS peer certificate on the heap with WOLFSSL_DYNAMIC_PEER_CERT - #11990

Merged
dgarske merged 2 commits into
wolfSSL:masterfrom
LinuxJedi:ossl-peercert-ref
Nov 5, 2026
Merged

dgarske merged 2 commits into
wolfSSL:masterfrom
LinuxJedi:ossl-peercert-ref

Conversation

@LinuxJedi

@LinuxJedi LinuxJedi commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Summary

With KEEP_PEER_CERT, which OPENSSL_EXTRA, OPENSSL_ALL and the
application recipes turn on, every WOLFSSL embeds a whole WOLFSSL_X509
for the peer's certificate (3.4 KB on Cortex-M33, 4.3 KB with
OPENSSL_ALL), and InitSSL() zeroes it. It is paid for even when there is
no peer certificate: servers that do not ask for client authentication, PSK
connections, and every object before its handshake.

This adds WOLFSSL_DYNAMIC_PEER_CERT. With it, ssl->peerCert is a pointer,
NULL until a certificate is kept, and the WOLFSSL holds one reference. A
new certificate (renegotiation, wolfSSL_clear() and a new handshake, a
certificate restored from a ticket) replaces the object instead of reusing
it, so a reference handed out stays valid, also after wolfSSL_free().
With X509 reference counting (OPENSSL_EXTRA or OPENSSL_EXTRA_X509_SMALL),
wolfSSL_get_peer_certificate() returns the kept object with a new
reference, as OpenSSL does, instead of a copy that parses the DER again.

Gating and defaults

  • On by default when KEEP_PEER_CERT and WOLFSSL_SMALL_STACK are defined,
    as large objects then already go on the heap (--enable-smallstack, or
    WOLFSSL_SMALL_STACK in user_settings.h). Default configure builds are
    unchanged.
  • NO_WOLFSSL_DYNAMIC_PEER_CERT keeps the embedded object.
  • Defining WOLFSSL_DYNAMIC_PEER_CERT turns it on without
    WOLFSSL_SMALL_STACK.
  • Always off with WOLFSSL_NO_MALLOC, even if defined, as the certificate
    could not be allocated and so would never be kept. Undefined without
    KEEP_PEER_CERT.
  • With the flag off the certificate stays embedded and behaves as before,
    including ReinitX509() and wolfSSL_get_peer_certificate() returning a
    copy.

The names (WOLFSSL_DYNAMIC_PEER_CERT, NO_WOLFSSL_DYNAMIC_PEER_CERT,
GetPeerCert(), FreePeerCert(), SetPeerCert()) match dgarske's
ossl_lowmem branch. Internal code reaches the certificate through
GetPeerCert(ssl) in either mode.

Other changes

  • The leaf is copied, and with the flag on allocated, once per handshake,
    in SetupStoreCtxCallback() or ProcessPeerCerts() (peerCertSet).
  • wolfSSL_get0_verified_chain() uses the kept certificate directly
    instead of taking a reference or copy and dropping it.
  • TLS 1.3 dual-algorithm CertificateVerify returns PEER_KEY_ERROR when no
    peer certificate is kept.
  • The wolfSSL_get_peer_certificate() Doxygen describes both modes.

Measurements

Memory harness, STM32U585 (Cortex-M33), ECC, no client auth on the server,
heap in bytes, before -> after. The harness settings define
WOLFSSL_SMALL_STACK, so the flag is on. Measured on the first version of
this change (#11925), which always used the heap pointer and matches this
PR with the flag on.

OPENSSL_EXTRA OPENSSL_ALL
sizeof(WOLFSSL) 4560 -> 1144 5576 -> 1304
server connection, resident 6842 -> 3426 (-50%) 12366 -> 8094 (-35%)
server wolfSSL_new() 7292 -> 3876 16916 -> 12644
client wolfSSL_new() 7726 -> 4310 14914 -> 10642
client peak, wolfSSL_new() + handshake 23694 -> 23692 34290 -> 34289
client connection, resident 13961 -> 13961 23276 -> 23276
flash 394440 -> 393152 409184 -> 407616

Client handshake and resumption times are unchanged. On x86_64,
sizeof(WOLFSSL) goes 5632 -> 1552 (EXTRA) and 6976 -> 1752 (ALL), and the
resident server connection 7986 -> 3906 and 13898 -> 8674.

Testing

  • make check with the flag on and off: --enable-opensslextra,
    --enable-opensslall, --enable-all, and
    --enable-opensslextra=x509small with -DKEEP_PEER_CERT.
  • Library builds: --enable-staticmemory with WOLFSSL_NO_MALLOC and the
    flag requested (it resolves off), --enable-smallstack (on by default),
    and NO_WOLFSSL_DYNAMIC_PEER_CERT (off).
  • New test_wolfSSL_get_peer_certificate_ref covers both modes: no
    certificate before the handshake, none kept by a server without client
    auth, reference counts with the flag on and separate copies with it off,
    and a returned certificate that outlives wolfSSL_clear() plus a new
    handshake and wolfSSL_free(). It needs OPENSSL_EXTRA, so the x509small
    builds skip it.

Related

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

MemBrowse Memory Report · Explore what changed →

gcc-arm-cortex-m4-openssl-compat

  • FLASH: .text -832 B (-0.1%, 800,484 B / 1,048,576 B, total: 76% used)

zephyr-frdm_rw612

  • FLASH: log_const_area -8 B, rodata -8 B, text +1,544 B (+0.5%, 283,992 B / 67,108,864 B, total: 0% used)

No memory changes detected for:


Catch memory regressions in CI with MemBrowse. Free for open source.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Shared peer certificates can trigger premature ex-data cleanup while retained references remain alive, and the new test incorrectly uses PEM loading for DER fixtures.

2 open issues
What changed in this PR

Adds optional heap allocation for TLS peer certificates, reducing per-connection memory use when no certificate is kept. This extends the earlier certificate-parsing optimizations while preserving embedded storage when the feature is disabled.

Changes:

  • Adds WOLFSSL_DYNAMIC_PEER_CERT, enabled by default for eligible small-stack builds.
  • Centralizes certificate ownership and access, returning shared references where supported.
  • Adds lifetime tests and documents reference-versus-copy behavior.
File Description
wolfssl/​wolfcrypt/​settings.h Defines feature defaults and allocation constraints.
wolfssl/​internal.h Adds optional pointer storage and certificate helpers.
tests/​api/​test_tls13.c Adapts ticket verification assertions for pointer storage.
tests/​api/​test_ssl_cert.h Registers the new certificate lifetime test.
tests/​api/​test_ssl_cert.c Tests ownership, reference counts, and certificate lifetime.
src/​tls13.c Uses certificate access helpers and checks for missing certificates.
src/​ssl.c Updates peer-name access and certificate clearing.
src/​ssl_api_cert.c Supports shared references and direct verified-chain access.
src/​internal.c Implements allocation, replacement, and release helpers.
doc/​dox_comments/​header_files/​ssl.h Documents ownership and required certificate cleanup.
.wolfssl_known_macro_extras Registers the feature and opt-out macros.

Review effort: Balanced

Comment thread src/ssl_api_cert.c Resolved
Comment thread tests/api/test_ssl_cert.c Resolved

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #11990

Scan targets checked: wolfcrypt-src, wolfcrypt-bugs, wolfssl-src, wolfssl-bugs
Not reviewed (no scan target covers them): .wolfssl_known_macro_extras, doc/dox_comments/header_files/ssl.h, tests/api/test_ssl_cert.h
Coverage: 7 of 9 in-scope changed file(s) opened by the reviewer; not opened: tests/api/test_ssl_cert.c, tests/api/test_tls13.c

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

Review tier: Lite

@dgarske dgarske self-assigned this Nov 4, 2026
@dgarske
dgarske self-requested a review November 4, 2026 20:17
dgarske
dgarske previously approved these changes Nov 4, 2026
@dgarske dgarske assigned LinuxJedi and unassigned dgarske Nov 5, 2026

@dgarske dgarske left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Jenkins CI seems unhappy with Valgrind and openssl

@LinuxJedi

Copy link
Copy Markdown
Member Author

The Jenkins failures on this run also hit other PRs since #11991 and #11992
merged, so they come from master rather than this change:

Both need a fix on master; this PR can be retested once those land.

…ER_CERT

With KEEP_PEER_CERT, which OPENSSL_EXTRA turns on, every WOLFSSL embeds
a whole WOLFSSL_X509 for the peer's certificate (3.4 KB on Cortex-M33,
4.3 KB with OPENSSL_ALL) and InitSSL() zeroes it. It is paid for when
there is no peer certificate: servers that do not ask for client
authentication, PSK connections, and objects before the handshake.

WOLFSSL_DYNAMIC_PEER_CERT makes ssl->peerCert a pointer instead, NULL
until a certificate is kept, with the WOLFSSL holding one reference.
SetPeerCert() replaces it with a new object each time instead of reusing
it, so a reference handed out stays valid across renegotiation,
wolfSSL_clear() and a certificate restored from a ticket, and after
wolfSSL_free(). With X509 reference counting (OPENSSL_EXTRA or
OPENSSL_EXTRA_X509_SMALL), wolfSSL_get_peer_certificate() then returns
the kept object with a new reference, as OpenSSL does, instead of a
copy that parses the certificate DER again.

WOLFSSL_DYNAMIC_PEER_CERT is on by default when KEEP_PEER_CERT and
WOLFSSL_SMALL_STACK are defined, as large objects then already go on
the heap. Define NO_WOLFSSL_DYNAMIC_PEER_CERT to keep the embedded
object, or WOLFSSL_DYNAMIC_PEER_CERT to turn it on without
WOLFSSL_SMALL_STACK. It is always off with WOLFSSL_NO_MALLOC, where the
certificate could not be allocated and so would never be kept. With the
flag off the certificate stays embedded and behaves as before.

GetPeerCert() returns the kept object in either mode, and FreePeerCert()
and SetPeerCert() release and replace it. The leaf is copied, and with
the flag on allocated, once per handshake (peerCertSet).
wolfSSL_get0_verified_chain() uses the kept object directly instead of
taking a reference or copy and dropping it.
ExternalFreeX509() ran the HAVE_EX_DATA_CLEANUP_HOOKS clean-up on every
wolfSSL_X509_free(), before the reference count was dropped, so freeing
one reference cleared the ex_data, and ran its clean-up routine, for every
other holder. With WOLFSSL_DYNAMIC_PEER_CERT this is reachable through the
peer certificate: freeing one returned reference, wolfSSL_clear() or
wolfSSL_free() cleared the ex_data of a certificate still in use. Run the
clean-up with the final free, as WOLFSSL_RSA, WOLFSSL_BIO and
WOLFSSL_X509_STORE do.

The peer certificate test now checks that ex_data survives an extra
reference being freed and wolfSSL_clear(), and is cleaned up once at the
last free. It also loads the server certificate with CERT_FILETYPE, to
match the certificate files test_memio_setup() selects.
@LinuxJedi

Copy link
Copy Markdown
Member Author

Jenkins: retest this please

@LinuxJedi
LinuxJedi requested a review from dgarske November 5, 2026 09:56
@LinuxJedi LinuxJedi removed their assignment Nov 5, 2026
@dgarske
dgarske merged commit 62282d6 into wolfSSL:master Nov 5, 2026
443 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants