Reduce hot-path allocations in IPC, Lua, and metrics

[popduke]

This removes temporary buffers and redundant copies from message processing and metrics collection while preserving queue publication, completion ownership, metric values and output formats.

Changes

• Unify Rust queue appends under try_write_with(exact_len, encoder, on_committed). Source submissions and completions encode directly into available queue space; full queues retain the original record for retry. Egress builds one complete record and copies it to each destination queue.
• Share read-only Lua callbacks within each VM, format uint64 values without temporary heap strings, and iterate message fields without an intermediate collection.
• Serialize JSON metric numbers as exact decimal strings without temporary strings, write Prometheus labels into the response buffer, move owned Plugin labels into snapshots, and filter requested metrics before SDK-to-Protobuf conversion. Sampling callbacks and cumulative aggregation still run as before.

Rust API change: try_write and try_write_observed are replaced by try_write_with. Callers with encoded bytes copy them into the supplied cursor. Encoding failures and panics do not publish a commit; the post-commit callback runs before notification. Queue bytes and the cross-language wire format are unchanged. The public IPC and SDK contracts describe the updated interface.

Measurements and limits

Selected local release measurements on macOS arm64, comparing each optimization with its own baseline:

Operation	Allocations before → after
JSON rendering, 256-channel fixture	2,572 → 1,033
Prometheus rendering, 256-channel fixture	9,240 → 10
Collect only the memory metric, 256-channel fixture	2,615 → 558

Source submission removes one record-buffer allocation per record. Egress removes one payload-buffer allocation per nonempty output; copying into each destination queue remains necessary.

The filtered collection benchmark was about 58% faster. Complete localhost HTTP requests for filtered Runner-local metrics were about 39–40% faster with 256 Pipeline-id label combinations. Those HTTP measurements include the client and server but exclude Pipeline/Plugin communication.

This is not a claim of universal speedup:

• A full-queue, 256-byte Source benchmark with 16 repeated space checks was slower in 8/9 initial rounds, with a +23% median. Follow-up controls identified repeated encoded-length calculation as a contributor, but did not fully explain the regression. No length cache was added.
• Some small Egress and unfiltered collection cases were approximately 1–3% slower. Unfiltered metrics conversion also requests 384 additional bytes for the outer vector in the measured fixture.
• These measurements do not establish production throughput, distributed tail latency, or lower RSS.

Validation

The final code passed tools/verify-rust.sh on macOS arm64 with Rust 1.97.1, including formatting, checks, Clippy, workspace/SDK tests, real Rust and Java plugins, HTTP metrics, cross-language vectors, Loom and doctests.

Focused coverage includes queue-full retries, encoder failure/panic/length mismatch, commit-before-notification behavior, writer termination during encoding, Lua integer and read-only behavior, and metrics output equivalence. Metrics tests compare complete Protobuf bytes from the same SDK snapshot and verify that empty filtered results still sample callbacks and preserve cumulative history. JSON and Prometheus HTTP samples were also parsed and checked.