Skip to content

Hold metadata text in three fixed blocks and let consumers choose its fields - #258

Merged
kahrendt merged 3 commits into
mainfrom
metadata-inline
Nov 11, 2026
Merged

kahrendt merged 3 commits into
mainfrom
metadata-inline

Conversation

@kahrendt

@kahrendt kahrendt commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

server/state metadata still copied every string onto the heap through operator new, which aborts on ESPHome. Sendspin/spec#355 caps title, artist, album_artist and album at 255 bytes and artwork_url at 2048, so the text now lives in fixed blocks allocated at start() and a metadata state allocates nothing. Part of Sendspin/backlog#92.

Behaviour change: a value over its cap is cut at a UTF-8 boundary, with one warning per connection. METADATA_STRICT (in protocol.cpp, false for now) later turns that into a protocol error that closes the connection, as #256 staged for group_id.

Change

  • Field opt-in: MetadataRoleConfig has one bool per text field (all true by default) and a buffer_location; add_metadata(MetadataRoleConfig config = {}) keeps the no-argument call. An unkept field is dropped at parse and takes no storage, e.g. {.artwork_url = false} saves 2,048 B per block.
  • Three text blocks (9,204 B with every field, 3,060 B without the URL), PSRAM-preferring, allocated at start(): the slot's two places and the main loop's current state. A start() that cannot allocate them fails like artwork's.
  • Hand-off: the protocol task copies an arrival's kept text into its slot place under the Inbox mutex (at most 3,068 B, the largest Inbox critical section). The main loop peeks at the timestamps, decides what is due with no lock held, and commits only if nothing arrived since; the public strings are filled from the current block outside the lock through ReservedText, reserved per kept field when the role is built.
  • Scheduled states wait in the slot instead of on the main loop. ScheduledStateSlot (src/scheduled_state_slot.h) holds that protocol, and the color role now uses it too, with unchanged behaviour. One interleaving shifts: an arrival during a callback replaces a pending state that falls due in that same callback, the same outcome as the arrival landing just before the drain.

Tests

  • Parse tables at both caps with Control: rows (cap, cap+1, 2/3/4-byte characters straddling it, lenient and strict) and a kept-fields table; an end-to-end check of kept and dropped fields.
  • MetadataSlot.ThreeBlocksCarryEveryStateToTheListenerInOrder: a white-box table asserting after every step that the three blocks are a permutation, plus the delivered and pending titles, including a re-peek, cleanup, re-entrant stop() and restart. ScheduledStateSlot tests cover the decision table, generations, deciding with the Inbox mutex free and the slot's dirty bit.
  • Alloc window for metadata states (an over-cap one, a scheduled one delivered later, one without text): 0. Mutants caught include a commit ignoring the sequence, a deadline decided under the lock, block aliasing, a missing re-peek, the same-tick loop removed and a settle/commit/reset polarity flip.
  • Gaps: the strict close path and a failed block allocation at start() need a production seam.

Stack (ESP-IDF 5.5.4, esp32): protocol task 6,512 B -Os / 6,624 B -Og (was 6,544 / 6,688), within its 7,168 B default; every other task unchanged. tools/stack_usage/README.md updated.

… fields

roles/metadata/v1.md caps title, artist, album_artist and album at 255
bytes and artwork_url at 2048 (Sendspin/spec#355), so server/state no
longer copies its strings onto the heap. Each state's text lives in one
of three PSRAM-preferring blocks allocated at start(): the slot's two
places and the main loop's current state. The protocol task copies an
arrival's kept text into its place under the Inbox mutex; the main loop
peeks at the timestamps, decides what is due with no lock held, commits
only if nothing arrived meanwhile, and fills the public strings from the
current block, reserved per kept field when the role is built.

A pending scheduled state now waits in the slot rather than on the main
loop, and the color role shares the same ScheduledStateSlot and drain.
MetadataRoleConfig lists which text fields to keep (all by default);
unkept fields are dropped at parse and take no storage. An over-cap
value is cut at a UTF-8 boundary with one warning per connection until
METADATA_STRICT turns it into a protocol error.
The per-section frame note now describes the color and controller
parsers, the slot paragraph names ScheduledStateSlot, the public
MetadataRoleConfig doc points at the integration guide for its costs,
and internals.md points at fill_place() and scheduled_state_slot.h
instead of restating them.
@kahrendt
kahrendt marked this pull request as ready for review November 11, 2026 12:37
@kahrendt
kahrendt requested a max review from Copilot November 11, 2026 12:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Reentrant drains can commit stale metadata or deliver state early, while lifecycle-test callbacks can destroy their own executing closures before accessing captures, causing undefined behavior.

5 open issues
What changed in this PR

Replaces per-message metadata text allocations with fixed storage allocated at startup, improving embedded reliability while letting consumers omit unused text fields.

Changes:

  • Adds MetadataRoleConfig field selection, memory placement, and UTF-8-safe text limits.
  • Introduces three reusable text blocks and shared scheduled-state handling for metadata and color.
  • Expands parser, lifecycle, and allocation tests; updates consumer and architecture documentation.
File Description
tools/​stack_usage/​README.md Updates measured protocol-task stack bounds.
tests/​test_role_deactivation.cpp Adapts cleanup checks to scheduled slots.
tests/​test_protocol.cpp Tests text caps, truncation, and field selection.
tests/​test_inbox.cpp Tests scheduled-slot decisions, generations, and locking.
tests/​test_encrypted_lifecycle.cpp Checks delivered fields, truncation, and pending metadata.
tests/​test_client_lifecycle.cpp Exercises three-block ownership, teardown, and restart.
tests/​test_allocations.cpp Adds zero-allocation checks for metadata states.
tests/​README.md Documents expanded test coverage.
tests/​CMakeLists.txt Configures and documents private test access.
src/​teardown_tracker.h Shares generation checks with peek-based drains.
src/​scheduled_state_slot.h Adds shared timestamp-based state storage and draining.
src/​protocol.cpp Parses bounded metadata text without owning strings.
src/​protocol_messages.h Defines metadata bounds, layouts, views, and outcomes.
src/​metadata_role.cpp Implements reusable text storage and public-state filling.
src/​metadata_role_impl.h Holds configuration, text blocks, and scheduled state.
src/​inbox.h Adds persistent slot read and settle operations.
src/​controller_role_impl.h Updates a comment distinguishing controller scheduling needs.
src/​connection.h Tracks the once-per-connection truncation warning.
src/​color_role.cpp Uses the shared scheduled-state drain.
src/​color_role_impl.h Replaces held state with a scheduled slot.
src/​client.cpp Allocates metadata blocks at startup and accepts configuration.
src/​client_dispatch.cpp Dispatches metadata parsing and handles truncation or rejection.
include/​sendspin/​metadata_role.h Updates construction and documents bounded text behavior.
include/​sendspin/​config.h Adds metadata field and memory-placement settings.
include/​sendspin/​client.h Exposes optional metadata configuration.
docs/​internals.md Describes scheduling, text ownership, and memory flow.
docs/​integration-guide.md Documents configuration, text limits, and storage costs.
docs/​conventions.md Documents scheduled-slot generation handling.
CLAUDE.md Updates architecture and allocation-test guidance.

Review effort: Max

Comment thread src/scheduled_state_slot.h Resolved
Comment thread tests/test_allocations.cpp Resolved
Comment thread tests/test_client_lifecycle.cpp Resolved
Comment thread src/client_dispatch.cpp Resolved
Comment thread src/metadata_role.cpp Resolved
… the extract exception

The MetadataSlot test's on_title fires through std::exchange so a lambda
no longer destroys itself while running, the allocation window no longer
compares live clock readings, the extract-then-release docs name the
metadata role's in-place copy, and the late-registration guard goes, as
client.h documents role registration before start().
@kahrendt
kahrendt merged commit ad20df2 into main Nov 11, 2026
16 checks passed
@kahrendt
kahrendt deleted the metadata-inline branch November 11, 2026 13:12
kahrendt added a commit that referenced this pull request Nov 11, 2026
## Summary

Documents that no callback may call `loop()`. Every main-loop callback
already runs inside one of the client's own calls (`loop()`, `start()`,
`stop()` or its destructor), and `loop()` does not support being
re-entered. The scheduled-state drain from #258 relies on this: a
callback that re-entered `loop()` between its peek and commit could
commit a stale decision. Closes #257.

## Change

- `include/sendspin/client.h`: the `loop()` doc states the rule and
points at the integration guide.
- `docs/integration-guide.md`, Thread Safety Summary: the rule and why,
including callbacks on other threads.
- `docs/conventions.md`: the re-entrant-teardown rule names `loop()` as
the one client method a callback does not call.
- `docs/internals.md`, "Re-entrant Teardown During Callback Dispatch":
the only drain nested beneath another is the one `stop()` runs.

## Tests

Documentation only; no behaviour changes, and nothing enforces the rule
at runtime (conventions.md "Contracts are documented and enforced by
documentation, not by defensive code").
kahrendt added a commit that referenced this pull request Nov 11, 2026
## Summary

First step of moving the public API from `std::string` to
`std::string_view`. Metadata and group text already live in fixed
storage (#256, #258); the public structs still copied it into
`std::string`s kept allocation-free only by `ReservedText` (about 3 KB
of internal RAM for metadata). The view forms hand consumers that
storage directly; the string forms are deprecated and go in v0.11.0 with
`ReservedText`. Part of Sendspin/backlog#92.

**Migration:** override `on_track_metadata(const TrackMetadata&)`
instead of `on_metadata()`, `on_group_state(const GroupState&)` instead
of `on_group_update()`, and call `group_state()` instead of
`get_group_state()`. Text fields become
`std::optional<std::string_view>`: copy them if you keep them past the
callback (or, for `group_state()`, past the next return to `loop()` or
`stop()`). A view is not guaranteed to be NUL-terminated, so print it
with `%.*s` or copy it before handing it to a C-string API.
docs/integration-guide.md "Deprecated String API" has the full table.

## Change

- **Metadata:** `TrackMetadata` views the role's current text block,
valid for the callback.
- **Group:** `GroupState` serves both the delta `on_group_state()`
receives (the fields changed by the group/update messages since the last
call) and the merged state `group_state()` returns. Both view an inline
copy the main loop keeps, so the views outlive the string forms'
removal.
- **Dispatch:** the library calls the new callback, then the deprecated
one; both default to no-ops, so a consumer overrides one. A `stop()`
inside the new callback skips the deprecated call (group: the drain
generation; metadata: `TeardownTracker::completed()`, since the protocol
task may bump the cleanup generation meanwhile).
- **Deprecated** with `@deprecated` and `[[deprecated("...; removed in
v0.11.0")]]`: `ServerMetadataStateObject`, `on_metadata()`,
`GroupUpdateObject`, `on_group_update()`, `get_group_state()`. A
consumer of only the new API compiles warning-free; the library silences
the warning only at its own compat sites.
- `get_track_progress_ms()` / `get_track_duration_ms()` read the role's
current state, so the removal can delete the deprecated object outright.
The examples use the new API.

## Tests

- The metadata and group content tests check both forms carry identical
content (cut text, unkept fields, cleared fields, delta versus merged
state) and that the views do not point into the deprecated strings.
- Listener-form tables: only new, only old, both (new first), a `stop()`
from the new callback, and a protocol-task teardown during it (pins
`completed()` over `accepts()`). The progress getters are pinned with
and without a listener.
- The metadata and group alloc windows stay at 0 with listeners
overriding both forms. Mutants caught include views into the deprecated
strings, the deprecated call skipped or reordered, the re-entrancy skip
removed, and the getters reading the deprecated object.
- Builds checked: `-DENABLE_WERROR=ON` with examples,
`-DSENDSPIN_ENABLE_METADATA=OFF`, and an ESP-IDF esp32s3 build with no
deprecation warnings from the library.

Next in the series: the pairing listener strings, server information and
token getters; then the vector callbacks, the persistence provider and
the configs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants