Skip to content

Large IOContext write buffers cause severe RSS growth under sustained muxing #328

Description

@JxnLexn

Summary

Sustained fMP4 muxing with large, variably sized FFmpeg writes can create severe process-heap retention under real long-lived camera workloads. In the affected camera.ui deployment, RSS reaches the 10 GiB container limit while V8 heap usage, file descriptors, threads, mappings, and socket queues remain bounded.

The hot allocation path is the fresh Napi::Buffer<uint8_t>::Copy() performed for every IOContext::WritePacket() callback. Large writes are copied through the regular Node/libc heap even though they are transient.

Affected workload

The production workload differs materially from a single synthetic transcoding stream:

  • Ubuntu 24.04.4 x86_64
  • glibc 2.39
  • Node.js 24.18.0
  • four permanent HKSV prebuffer streams
  • H.264 + AAC remux where the input codec is already supported
  • fragDuration: 4_000_000 (4 seconds)
  • default bufferSize: 2 * 1024 * 1024
  • high-resolution/high-bitrate camera input, producing large and variably sized muxer writes

No MALLOC_* tunables are set in production.

Long-run evidence

A production run started on August 19, 2026 at 18:36 CEST. Proxmox RRD data shows the container below 1 GiB around 19:30, then approximately 1.6 GiB at 20:00, 5.2 GiB at 20:30, 9.4 GiB at 21:00, and the 10 GiB cgroup limit by 21:30. It remained pinned at the limit by reclaim rather than plateauing naturally.

At 00:16 CEST the HomeKit process alone reported:

  • RSS: 9,360,356 KiB
  • private dirty: 9,360,352 KiB
  • anonymous: 9,360,352 KiB
  • swap: 0

The cgroup had accumulated more than 53 million memory.high events, with no kernel OOM kill. A preceding run had independently reached the same limit before the service was restarted.

An isolated FMP4Stream against the same real camera source reproduced native post-GC growth without HomeKit/HAP or application queues. Short allocator A/B runs with MALLOC_MMAP_THRESHOLD_=65536 or a low trim threshold reduced that growth substantially. These short isolated runs identify the large-copy allocation path as a contributor; the production RRD history supplies the multi-hour non-plateau evidence.

Expected behavior

Large transient write buffers should not keep driving the regular process heap high-water mark under long-lived muxing. Their backing storage should be visible to V8's external-memory accounting and returned to the OS when the last JavaScript reference is collected, while preserving the required copy and callback-buffer lifetime semantics.

Proposed fix

Use page-backed external Node buffers for writes of at least 64 KiB, keep the existing Buffer::Copy() path for small writes and allocation fallback, account the external mapping with napi_adjust_external_memory, and release it from the N-API finalizer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions