Repository navigation
fix(mqtt): 8 KB MQTT task stack when TLS verifies a CA certificate - #3301
Conversation
|
this is also a valid bug. I haven't tested but can see on board with PSRAM + TLS + CA using BearSSL EC , it will crash EMS-ESP. It does use up an additional 3kb of heap though. @MichaelDvP do you approve this change? |
|
This was the original bug I was chasing when I found the others. With an EC CA set for certificate validation (Let's Encrypt ISRG Root X2), my BBQKees Gateway E32 V2 boot looped, crashing on every MQTT connect. It has run with this change since 2026-10-13 without a crash. |
|
Thanks for the solid diagnosis and measurements - the EC-chain overflow on the 5120-byte MQTT task is real, and 8192 is the right size when CA verification is in play. One thought on the cost: the +3 KB of internal RAM hits every PSRAM board, while most users run insecure TLS or no TLS and never need that stack. The task size is fixed at create time today, but we could size it from the config instead:
That way insecure / no-CA stays at 5120, and only CA users pay the extra 3 KB. Same peak headroom you measured (~2.1 KB on ISRG Root X2). would that work if we implement it that way? |
|
Yes, that works. I checked the insecure case first, as BearSSL still verifies the server's key exchange signature against the leaf key when rootCA is "insecure". Peak MQTT task stack use after one handshake with rootCA "insecure":
So with a CA it's the chain validation that needs the bigger stack, and insecure is fine for P-256 and P-384 server keys. A P-521 server key would still overflow 5120 in insecure mode. That's rare (Let's Encrypt doesn't issue P-521), but a self-signed broker cert could use it. Two ways to cover it:
I'd lean towards 1 as it's simpler and has no gap, but either is fine by me. Happy to update this PR whichever way you prefer. |
|
how about: 1.
|
On boards with PSRAM the MQTT client runs in its own task with a 5120 byte stack. Verifying an EC certificate chain against a CA, such as Let's Encrypt ISRG Root X2, needs about 6 KB, so the task overflowed and the board boot looped on every connect. The espMqttClient constructors now take the stack size, and startClient() passes EMSESP_MQTT_STACKSIZE_CA (8192) when TLS is on with a CA, keeping the 5120 default for insecure TLS and plain MQTT. The client is recreated when that changes. The stack usage log reports against the size the task was created with.
1d620af to
cdf6375
Compare
|
Implemented your suggestion and force-pushed: the Tested on the Gateway E32 V2 by switching rootCA between the X2 CA and "insecure" without a restart, twice each way: 8192 with 2152 bytes free on the CA, 5120 with 624 free on insecure, a clean reconnect every time and no restarts. Also rebased on dev; the details are in the description. |
MichaelDvP
left a comment
There was a problem hiding this comment.
This is imho a very good solution.
Problem
On boards with PSRAM the MQTT client runs in its own task with a 5120 byte stack (
EMC_TASK_STACK_SIZE). With a CA certificate configured, BearSSL verifies the broker's certificate chain inside that task. For an EC chain, such as Let's Encrypt's ISRG Root X2, that needs about 6 KB, so the task overflows and the ESP32 panics on every connect: a boot loop that only stops when MQTT or TLS is turned off.Fix
Size the MQTT task stack from the TLS settings, as suggested in review:
espMqttClient/espMqttClientSecureconstructors take a stack size, defaulting toEMC_TASK_STACK_SIZE(5120)MqttSettingsService::startClient()passesEMSESP_MQTT_STACKSIZE_CA(8192) when TLS is on androotCAis a certificate, and keeps the default for insecure TLS and plain MQTT. The client is recreated when that changes, not only when TLS is toggledMqttClient.cppreports against the size the task was created withBoards without PSRAM don't create the task (the client runs on the loop task, 8 KB), so nothing changes for them.
Testing
Peak stack use, each read after a fresh boot whose only handshake used the listed chain:
With
rootCAset to "insecure" (one handshake, test server):So a P-521 broker certificate would still overflow the default in insecure mode. That's rare (Let's Encrypt doesn't issue P-521) and left as it is.
With this change, on a Gateway E32 V2 against a Let's Encrypt broker (P-256 leaf), switching
rootCAin the settings without a restart, twice each way:Every switch recreated the client and reconnected, with no restart. Insecure mode against this broker peaks at ~4.5 KB of the 5120, which is the same as before this change.
s_16M_P,c3_mini_4M,nativeandnative-test, clang-format clean insrc/.Related MQTT TLS fixes, independent of each other: #3300, #3302, #3303.
Checklist
devbranch.clang-formatCHANGELOG_LATEST.mdupdated