xll-gen/shm

SimpleIPC is a high-performance, low-latency shared-memory IPC library connecting C++ (Host) and Go (Guest). It uses a lock-free, direct slot exchange model to achieve sub-microsecond latency.

This project is currently in beta and under active development. APIs and memory layouts may still change between minor versions. The current release version lives in the VERSION file and in the newest CHANGELOG.md entry — it is deliberately not restated here (AGENTS.md §Codebase Authority: VERSION is the release-version SSOT and the only place the number is written by hand). Version strings that do appear below are provenance labels on measurements and features ("measured on v0.8.9"), not the current version, and must not be bumped.

Features

• Low Latency: Uses atomic spin-loops with adaptive backoff (Spin -> Yield -> Sleep) to minimize OS scheduler overhead.
• Direct Mode: 1:1 Thread-to-Slot mapping eliminates contention and queuing delays.
• Zero Copy: Data is written directly to shared memory slots.
• Windows-Native: Built directly on Win32 primitives (CreateFileMapping / CreateEvent); compiles with MSVC and MinGW.
• Protocol Agnostic: Transmits raw bytes with a minimal 8-byte Transport Header for request matching.
• Guest-to-Host Calls: Supports async notifications from Go to C++.
• Header-Only C++: Easy integration via include/shm.

Performance Highlights

The project's "Direct Exchange" IPC mode significantly outperforms traditional methods, showcasing sub-microsecond latency and high throughput. This is achieved through a 1:1 thread-to-slot mapping, zero-copy operations, adaptive hybrid waiting, SMT-sibling thread affinity, and a held-slot session path that re-arms without a per-round-trip claim.

Measured on an AMD Ryzen 9 3900X (12C/24T, native Windows 11), benchmarks/harness.ps1 -HighPriority, AffinityAuto→Sibling, best-of-3, current benchmark (v0.8.9; the Direct-Exchange table is the v0.8.8-round measurement — v0.8.9 changed the guest-call and streaming defaults only). Numbers are round-trips per second (higher is better); single-thread RTT is ~70 ns (sub-100 ns).

Direct Exchange — request/response ping-pong:

Threads	64 B payload	1024 B payload
1	14.0 M ops/s	9.7 M ops/s
4	35.4 M ops/s	21.9 M ops/s
8	58.4 M ops/s	39.3 M ops/s

(Absolute peak observed: 77.7 M ops/s at 12 threads, 2026-07-09 scaling-curve session.)

Guest Call — Go→C++ push (e.g. RTD updates), 64 B echo: ~5.2 M ops/s (193 ns RTT) at 1 thread with the worker and a dedicated sender co-located on one physical core's SMT pair (HostConfig::guestWorkerAffinity + Go PinCurrentGoroutine, v0.8.9); ~2.1–2.7 M unpinned (and far noisier — scheduler placement lottery).

Streaming — bulk transfer: plateaus at ~3.6 GB/s with 4–8 MiB chunks; small/medium streams gain +28…+70% at 4T/8T from per-worker slot co-location (v0.8.7).

What actually makes it fast — the three biggest measured factors

1. Never touch the kernel on the hot path. Every request/response doorbell is elided via a two-sided Dekker handshake (peer publishes a WAITING flag before parking; the sender fires the SetEvent syscall only when it reads that flag), so two spinning peers exchange messages entirely in user space. Applying this to the guest-call path was the single largest measured jump in the project's history: +836% (≈9.4×) in one release (v0.8.6). A single stray syscall (~µs) dominates any number of nanosecond-scale optimizations.
2. Placement is a first-class optimization. Pinning the two endpoints of a slot to the two SMT siblings of one physical core (shared L1d) — instead of letting the scheduler place them — won +24…74% on Direct Exchange (v0.8.2, the AffinityAuto default), +28…70% on streaming (v0.8.7), and +88.8% on guest calls (v0.8.9). Unpinned runs also scatter by up to 86% between runs; co-location buys determinism as much as speed.
3. No per-round-trip claim machinery when it has no counterparty. The slot claim's gen-bump / CAS / lease-heartbeat exist for crash-recovery reclamation, which is opt-in and off by default. The held-slot session API (host, v0.8.5) and the fastPathAllowed-gated fast paths (guest, v0.8.8/9) skip them when the host attests reclaim is off — +18%…+41.5% per side on the 64 B cell, with a safe-by-default flag polarity so any version/config mismatch degrades to the slow path, never to corruption.

Honorable mention: several of the largest raw deltas in the history (+149% @8T among them) came from fixing the benchmark itself — the old harness added ~90 ns/op of its own timer/counter overhead, its instrumented counters were globally contended, and the old guest-call listener polled at the 15.6 ms Windows timer quantum. Numbers before those fixes measure the harness, not the library.

Note on older figures: results before shm v0.7.12 are not comparable — the benchmark harness itself was overhauled in 2026-07 (see above), and baseline discontinuities are marked in BENCHMARK_RESULTS.md.

For the full matrix, methodology, per-release A/B deltas, and Guest Call / streaming scenarios, see BENCHMARK_RESULTS.md and EXPERIMENTS.md.

Architecture

The library operates in Direct Mode, where a fixed pool of "Slots" is allocated in shared memory.

• Host (C++): Creates the shared memory region and manages the slot pool. It acts as the initiator of requests.
• Guest (Go): Attaches to the shared memory and processes requests. Each worker goroutine is pinned to a specific slot.

Memory Layout

The shared memory region consists of:

1. Exchange Header (64 bytes): Global metadata (Magic, Version, number of slots, slot size).
2. Slot Array: An array of Slots.

Each Slot (128-byte Header + Payload) contains:

• SlotHeader: Atomic state variables (State, HostState, GuestState) and message metadata (ReqSize, MsgSeq, MsgType).
• Request Buffer: Area where Host writes data.
• Response Buffer: Area where Guest writes data.

Synchronization

State transitions are handled via std::atomic (C++) and sync/atomic (Go).

• SLOT_FREE -> Host claims -> SLOT_BUSY -> Host writes -> SLOT_REQ_READY
• Guest sees SLOT_REQ_READY -> Processes -> Writes Response -> SLOT_RESP_READY
• Host sees SLOT_RESP_READY -> Reads Response -> SLOT_FREE

If a peer is not responsive (spinning times out), the other peer will wait on a named OS event (Semaphore/Event) to save CPU.

Usage

C++ Host

#include <shm/DirectHost.h>

shm::DirectHost host;
shm::HostConfig config;
config.shmName = "MyIPC";
config.numHostSlots = 4;
config.payloadSize = 1024 * 1024; // 1MB payload per slot
config.numGuestSlots = 0; // Set to >0 to enable Guest Calls

if (!host.Init(config).IsSuccess()) {
    std::cerr << "Failed to init host" << std::endl;
    return -1;
}

std::vector<uint8_t> resp;
// Send 4 bytes to any available slot
// Note: This blocks until response is received.
auto result = host.Send((const uint8_t*)"test", 4, shm::MsgType::NORMAL, resp);
if (result.HasError()) {
    // Handle error
}

Zero-Copy (FlatBuffers)

To send FlatBuffers without copying the data, use the ZeroCopySlot helper:

// 1. Acquire a Zero-Copy Slot
auto slot = host.GetZeroCopySlot();

// 2. Build FlatBuffer directly in shared memory
// slot.GetReqBuffer() returns the pointer to the buffer
flatbuffers::FlatBufferBuilder builder(slot.GetMaxReqSize(), nullptr, false, slot.GetReqBuffer());
// ... build your object ...

// 3. Send Request
// Signals MSG_TYPE_FLATBUFFER and handles negative size internally
// Returns Result<void>
auto res = slot.SendFlatBuffer(builder.GetSize());
if (res.HasError()) { /* Handle Error */ }

// 4. Access Response Directly (Zero-Copy)
uint8_t* respData = slot.GetRespBuffer();
int32_t respSize = slot.GetRespSize();

Go Guest

First, install the module:

go get github.com/xll-gen/shm

Then import it:

package main

import "github.com/xll-gen/shm/go"

func main() {
    // Basic Connection
    client, _ := shm.ConnectDefault("MyIPC")

    // Or Advanced Configuration
    /*
    client, _ := shm.Connect(shm.ClientConfig{
        ShmName: "MyIPC",
        ConnectionTimeout: 5 * time.Second,
    })
    */

    // Handler now receives msgType and returns msgType
    client.Handle(func(req []byte, respBuf []byte, msgType shm.MsgType) (int32, shm.MsgType) {
        if msgType == shm.MsgTypeFlatbuffer {
            // "req" automatically points to the FlatBuffer data
            // (even if it was sent with negative size alignment)
            // processFlatBuffer(req)
        }

        // Process req, write to respBuf
        // Return number of bytes written and the response type
        return int32(copy(respBuf, req)), msgType // Echo Type
    })

    client.Start()
    client.Wait()
}

Application Specific Message Types

You can use custom message types to multiplex different types of operations on the same connection. The system reserves types 0 through 127. User-defined types should start at MSG_TYPE_APP_START (128).

C++ Host:

#include <shm/IPCUtils.h>

// Define your custom Type
const uint32_t MY_OP_TYPE = (uint32_t)shm::MsgType::APP_START + 1;

// Send
host.Send(payload, size, (shm::MsgType)MY_OP_TYPE, resp);

Go Guest:

const MyOpType = shm.MsgTypeAppStart + 1

client.Handle(func(req []byte, respBuf []byte, msgType shm.MsgType) (int32, shm.MsgType) {
    if msgType == MyOpType {
        // Handle custom op
        return 0, MyOpType
    }
    // ...
})

Guest Call (Async)

The library supports Guest-initiated calls (e.g., for async callbacks). Specific slots are reserved for this purpose.

C++ Host (Listener):

shm::HostConfig config;
config.shmName = "MyIPC";
config.numHostSlots = 4;
config.numGuestSlots = 2; // 2 Async Slots
host.Init(config);

// Start background worker for Guest Calls
host.Start([](const uint8_t* req, int32_t reqSize, uint8_t* resp, uint32_t maxRespSize, shm::MsgType msgType) -> int32_t {
    if (msgType == shm::MsgType::GUEST_CALL) {
         // Process Guest Request
    }
    return 0; // Return response size
});

// To stop:
// host.Stop();

Go Guest (Caller):

// Send Guest Call
// msgType can be shm.MsgTypeGuestCall or custom
resp, err := client.SendGuestCall([]byte("AsyncData"), shm.MsgTypeGuestCall)

Request Buffer Capacity

Both sides can read the per-slot request-buffer capacity without holding a slot. The Host lays out every slot (host and guest alike) from the one geometry it publishes in the ExchangeHeader, so the value is identical for all slots and fixed for the lifetime of the mapping. It is the raw capacity — subtract your own framing before sizing a payload.

budget := client.MaxRequestSize() // == len(slot.RequestBuffer()); 0 if not connected

int32_t budget = host.GetMaxReqSize(0); // or slot.GetMaxReqSize() on a held/zero-copy slot

Large Data Streaming (Double Buffering)

For sending large datasets (exceeding slot size) efficiently, the library provides a Streaming API. This API splits the data into chunks and uses multiple slots in parallel ("Double Buffering" or "N-Buffering") to maximize throughput.

C++ Host

Use shm::StreamSender to send large data:

#include <shm/DirectHost.h>
#include <shm/Stream.h>

shm::DirectHost host;
host.Init(config);

shm::StreamSender sender(&host);
std::vector<uint8_t> bigData(10 * 1024 * 1024); // 10MB

// Send data with Stream ID 12345
auto result = sender.Send(bigData.data(), bigData.size(), 12345);
if (result.HasError()) {
    // Handle error
}

Go Guest

Use shm.NewStreamReassembler to handle streams:

package main

import "github.com/xll-gen/shm/go"

func main() {
    guest, _ := shm.NewDirectGuest("MyIPC")

    // Define stream handler
    onStream := func(streamID uint64, data []byte) {
        fmt.Printf("Received stream %d: %d bytes\n", streamID, len(data))
    }

    // Wrap your existing handler or use fallback
    handler := shm.NewStreamReassembler(onStream, myNormalMsgHandler)

    guest.Start(handler)
    guest.Wait()
}

Handling Long-Running Operations (Async Call Pattern)

The default timeout for operations is 10 seconds. For operations that may exceed this duration, or for asynchronous workflows, do not block the IPC channel. Instead, use the following pattern:

1. Host sends a Request (e.g., START_LONG_JOB).
2. Guest receives the request, starts the job in a background goroutine, and immediately returns an acknowledgement (Ack).
3. Host receives the Ack and is free to process other tasks.
4. When the job completes, the Guest sends the result back to the Host using a Guest Call (SendGuestCall).
5. Host processes the result via the handler registered in Start().

This ensures the 1:1 slot mapping remains available for high-frequency messages and prevents timeouts.

Nested IPC & Recursion

The library supports recursive calls (e.g., calling GetZeroCopySlot or Send while already holding a slot) provided that sufficient slots are available.

Important: If you plan to use nested IPC (e.g. Host -> Guest -> Host or recursive Host calls), you must configure numHostSlots to be at least N_threads * (Depth + 1).

• Example: If you have 1 thread performing a nested call (Depth 1), you need at least 2 slots.
• Failure to do so will result in Deadlock (the inner call waiting forever for a slot held by the outer call).

For complex recursion, it is recommended to double the slot count to provide a safety margin.

Note on Corruption: "Corruption" during nested calls usually stems from the application mistakenly reusing the same ZeroCopySlot object instance for the inner call. Always call GetZeroCopySlot() again to acquire a distinct slot for the nested operation.

Building

Requirements

• Windows: MSVC 2019+ or MinGW (GCC)
• Go: 1.18+
• CMake: 3.10+

Build Steps

The project uses Taskfile for automation (requires Task).

# Run all benchmarks (Builds C++ and Go, runs tests)
task run:benchmark

Windows (Manual with MSVC)

1. Generate Visual Studio Solution:

   mkdir build
   cd build
   cmake -S .. -B . -DSHM_BUILD_BENCHMARKS=ON

2. Build with Release Configuration:

   cmake --build . --config Release

3. Run Benchmarks: Use Taskfile or run the executables directly from the Release folder.

   task run:benchmark

Benchmarks

The benchmarks folder contains the C++↔Go latency/throughput harness. On Windows, benchmarks/harness.ps1 sweeps a threads × payloads matrix and writes results/<timestamp>/summary.md; pass -HighPriority for stable native numbers, -Profile stream for streaming, and -Mode guest-call for the Go→C++ push path.

# Windows (native): quick matrix, best of 3, high priority
pwsh -File ./benchmarks/harness.ps1 -Profile quick -Repeats 3 -HighPriority

# Portable helper script
./benchmarks/run.sh

See the Performance Highlights above for headline numbers and BENCHMARK_RESULTS.md for the full history.

Experiments

The experiments folder contains standalone latency tests (pingpong) used to validate the underlying synchronization primitives without the library overhead.

Documentation

• AGENTS.md: Developer guidelines and constraints.
• SPECIFICATION.md: Protocol details and memory layout.
• Source code is fully documented with Doxygen (C++) and GoDoc (Go) comments.

License and Third Party Notices

This project is licensed under the GPLv3 License. See LICENSE for details.

This project uses third-party open source software. For a list of third-party dependencies and their licenses, please see THIRD_PARTY_NOTICES.md.