[Arpan0995]

Description

With --enable-curve25519=nonblock (WC_X25519_NONBLOCK and WOLFSSL_ASYNC_CRYPT_SW) and an async device set on the WOLFSSL object, TLSX_KeyShare_GenX25519Key() gives the X25519 key a non-blocking context. TLSX_KeyShare_FreeAll() frees that context in the X25519 branch (since #9721), but the hybrid branch does not free it for the X25519 key of an X25519MLKEM768 (or other X25519 hybrid) key share. So the context leaks whenever such a key share is freed while it still holds its X25519 key: on the client when the server picks another group (after a HelloRetryRequest, or by choosing another key share the client sent), and in some failed handshakes on either side, such as the ones in the test below.

This frees it in the hybrid branch, in the same way as in the X25519 branch.

Testing

With an async device, the X25519 low-order cases in test_tls13_mlkem_bad_key_share (from #11764) make two X25519MLKEM768 handshakes fail on purpose, one rejected by the server and one by the client. On master they leak three of these contexts: the client's in both handshakes, and the server's in the one the server rejects.

./configure --enable-curve25519=nonblock --enable-debug --enable-trackmemory --disable-shared
make
./tests/unit.test --api -test_tls13_mlkem_bad_key_share

• On master this lists three Leak: ... Size 528 allocations (528 is sizeof(x25519_nb_ctx_t) here), prints WOLFSSL_TRACK_MEMORY: currentBytes after cleanup is 1584 and exits with status 131 (MEMORY_E). With this change it reports success.
• With --enable-curve25519=nonblock --enable-debug --disable-shared, macOS leaks --atExit on the same test reports three leaks on master, all from TLSX_KeyShare_GenX25519Key(), and none with this change. The same holds for the nonblock-sp-c64 configuration from os-check-linux.json, built with --disable-shared.
• A client that offers X25519MLKEM768 and secp256r1 to a server that accepts only secp256r1, both with an async device, completes the handshake on secp256r1, either after a HelloRetryRequest or directly when the client also sent a secp256r1 key share. On master leaks reports one leaked context on the client in both cases, and none with this change.
• An ASan/UBSan build of the nonblock configuration passes --api with no sanitizer reports, and the nonblock-sp-c64 configuration passes --api. --enable-all --enable-all-quantum-crypto builds with no warnings and passes the tls13 group.
• Tested on macOS (arm64) with Apple clang 21.

Checklist

• added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation

[Copilot]

🟢 Approval recommended

The narrow fix follows the existing allocation ownership and cleanup pattern, with no blocking issues identified.

No open issues 🎉

What changed in this PR

Fixes a memory leak in TLS hybrid key-share cleanup when an X25519 key retains its non-blocking context, such as after a failed handshake or selection of another group.

Changes:

• Frees the X25519 non-blocking context before releasing the hybrid key.
• Uses the same feature guards and cleanup pattern as standalone X25519 key shares.

File	Description
src/​tls.c	Adds non-blocking X25519 context cleanup to the hybrid key-share branch.

_{Review effort: Balanced}

[dgarske]

Okay to test. Contributor agreement on file

[github-actions]

retest this please

[github-actions]

MemBrowse Memory Report

gcc-arm-cortex-m0plus

• FLASH: .text +4 B (+0.0%, 70,431 B / 262,144 B, total: 27% used)

gcc-arm-cortex-m3

• FLASH: .text +24 B (+0.0%, 130,741 B / 262,144 B, total: 50% used)

gcc-arm-cortex-m4

• FLASH: .text +64 B (+0.0%, 209,948 B / 262,144 B, total: 80% used)

gcc-arm-cortex-m4-crypto-only

• FLASH: .text +64 B (+0.0%, 182,463 B / 262,144 B, total: 70% used)

gcc-arm-cortex-m4-dtls13

• FLASH: .text +64 B (+0.0%, 194,264 B / 1,048,576 B, total: 19% used)

gcc-arm-cortex-m4-openssl-compat

• FLASH: .rodata +8 B, .text +1,088 B (+0.1%, 796,012 B / 1,048,576 B, total: 76% used)

gcc-arm-cortex-m4-pkcs7

• FLASH: .text +64 B (+0.0%, 223,552 B / 262,144 B, total: 85% used)

gcc-arm-cortex-m4-pq

• FLASH: .text +64 B (+0.0%, 311,716 B / 1,048,576 B, total: 30% used)

gcc-arm-cortex-m4-rsa-only

• FLASH: .text +64 B (+0.0%, 343,064 B / 1,048,576 B, total: 33% used)

gcc-arm-cortex-m4-tls12

• FLASH: .text +64 B (+0.0%, 131,537 B / 262,144 B, total: 50% used)

gcc-arm-cortex-m4-tls13

• FLASH: .text +64 B (+0.0%, 249,230 B / 262,144 B, total: 95% used)

gcc-arm-cortex-m7

• FLASH: .text +64 B (+0.0%, 209,948 B / 262,144 B, total: 80% used)

gcc-arm-cortex-m7-pq

• FLASH: .text +64 B (+0.0%, 312,612 B / 1,048,576 B, total: 30% used)

gcc-arm-cortex-m7-tls13

• FLASH: .text +64 B (+0.0%, 249,294 B / 262,144 B, total: 95% used)

linuxkm-pie

• Data: __patchable_function_entries +64 B (+0.2%, 29,048 B)

linuxkm-standard

• Data: __patchable_function_entries +72 B (+0.1%, 52,040 B)

stm32-sim-stm32h753

• FLASH: .text +128 B (+0.1%, 205,552 B / 2,097,152 B, total: 10% used)
  No memory changes detected for:
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-sp-math

[philljj]

Leak reproduced and confirmed fixed. Thank you!

before:

total   Allocs   =      3180                                                    
total   Deallocs =      3177                                                    
total   Bytes    =   3563805                                                    
peak    Bytes    =     77500                                                    
current Bytes    =      1584                                                    
Leak: Ptr 0x2e635110, Size 528                                                  
Leak: Ptr 0x2e635350, Size 528                                                  
Leak: Ptr 0x2e636180, Size 528                                                  
  End Group  : tls13                                                            
 End API Tests

after:

total   Allocs   =      3180                                                    
total   Deallocs =      3180                                                    
total   Bytes    =   3563806                                                    
peak    Bytes    =     75916                                                    
current Bytes    =         0                                                    
  End Group  : tls13                                                            
 End API Tests                                                                  
 Failed/Skipped/Passed/All: 0/0/1/1                                             
wolfSSL Entering wolfSSL_Cleanup

Tests good with multi-test with gcc-c89, clang-tidy, tls13, dtls13, etc scenarios.