You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Substantial overhead caused by encoding messages for logging #215
I'm working on a multiprocess branch that passes a lot of data back and forth between server and client.
When looking at a flamegraph, it's clear that >50% of my client's cpu time is spent on logging (encoding) sends. Likewise for the server with responses.
The culprit is toString() from the params/results/request/response, which ends up in kj::_::encodeCEscapeImpl(), which is horribly slow.
Current master:
Fixed:
As you can see, before removing the logging, 52% of my thread's runtime is spent in serverInvoke (which calls foo.toString(), which calls structString(), which calls kj::_::encodeCEscapeImpl(). With the debug lines removed, serverInvoke is no longer visible in the flamegraph at all.
I would imagine the sv2 implementation suffers from this as well, though I'm not sure how chatty it is, so I can't speak to the potential impact.
I'll open a RFC PR which fixes the issue rather naively. A more robust solution would introduce logging categories similar to Bitcoin Core's.
Edit: See #216.
Also, toned down the original title :)
changed the title [-]Extreme overhead caused by encoding messages for logging[/-][+]Substantial overhead caused by encoding messages for logging[/+]on Sep 23, 2025
Activity
theuni commentedon Sep 23, 2025
Ping @Sjors, you might be seeing this as well.
[-]Extreme overhead caused by encoding messages for logging[/-][+]Substantial overhead caused by encoding messages for logging[/+]Sjors commentedon Sep 24, 2025
Do I understand correctly that this "only" impacts Bitcoin Core when running with
-debug=ipc?theuni commentedon Sep 24, 2025
@Sjors No, that's the issue I'm afraid.
The stringification happens in Core regardless of the selected logging level, and on the client even if no log callback is set.
Sjors commentedon Sep 24, 2025
Ah, ok that sounds more urgent then. I'll go and review your fix.
Merge bitcoin-core#220: Add log levels and advertise them to users vi…
Merge #33517: multiprocess: Fix high overhead from message logging
Merge bitcoin/bitcoin#33517: multiprocess: Fix high overhead from mes…
Merge bitcoin/bitcoin#33517: multiprocess: Fix high overhead from mes…
Merge bitcoin/bitcoin#33517: multiprocess: Fix high overhead from mes…