Skip to content

Substantial overhead caused by encoding messages for logging #215

Closed
bitcoin/bitcoin
#33517

Description

@theuni

I'm working on a multiprocess branch that passes a lot of data back and forth between server and client.

When looking at a flamegraph, it's clear that >50% of my client's cpu time is spent on logging (encoding) sends. Likewise for the server with responses.

The culprit is toString() from the params/results/request/response, which ends up in kj::_::encodeCEscapeImpl(), which is horribly slow.

Current master:
Image
Fixed:
Image

As you can see, before removing the logging, 52% of my thread's runtime is spent in serverInvoke (which calls foo.toString(), which calls structString(), which calls kj::_::encodeCEscapeImpl(). With the debug lines removed, serverInvoke is no longer visible in the flamegraph at all.

I would imagine the sv2 implementation suffers from this as well, though I'm not sure how chatty it is, so I can't speak to the potential impact.

I'll open a RFC PR which fixes the issue rather naively. A more robust solution would introduce logging categories similar to Bitcoin Core's.

Edit: See #216.
Also, toned down the original title :)

Activity

  1. theuni commented on Sep 23, 2025

    @theuni
    ContributorAuthor

    Ping @Sjors, you might be seeing this as well.

  2. changed the title [-]Extreme overhead caused by encoding messages for logging[/-] [+]Substantial overhead caused by encoding messages for logging[/+] on Sep 23, 2025
  3. Sjors commented on Sep 24, 2025

    @Sjors
    Member

    Do I understand correctly that this "only" impacts Bitcoin Core when running with -debug=ipc?

  4. theuni commented on Sep 24, 2025

    @theuni
    ContributorAuthor

    @Sjors No, that's the issue I'm afraid.

    The stringification happens in Core regardless of the selected logging level, and on the client even if no log callback is set.

  5. Sjors commented on Sep 24, 2025

    @Sjors
    Member

    Ah, ok that sounds more urgent then. I'll go and review your fix.

  6. added a commit that references this issue on Oct 2, 2025
    ec86e43
  7. added a commit that references this issue on Oct 14, 2025
  8. added a commit that references this issue on Jun 28, 2026
  9. added a commit that references this issue on Jun 30, 2026
  10. added a commit that references this issue on Aug 19, 2026
  11. locked and limited conversation to collaborators on Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions