Skip to content

feat: add durable producer flush - #95

Draft
infiniteregrets wants to merge 2 commits into
mainfrom
m/producer-flush
Draft

feat: add durable producer flush#95
infiniteregrets wants to merge 2 commits into
mainfrom
m/producer-flush

Conversation

@infiniteregrets

Copy link
Copy Markdown
Member

Summary

  • add Producer.flush() as a durable, non-terminal boundary for prior submissions
  • force the current partial batch without waiting for linger, then wait for every prior record acknowledgement
  • keep later submissions outside the boundary and preserve producer reuse
  • make terminal submit/ack failures sticky across tickets, submit(), flush(), and close()
  • keep an ordered flush running if its caller is cancelled
  • document the new checkpoint in the centralized producer example

Why

Previously, closing the producer was the only way to force a partial batch and wait for durability. Applications need an explicit durability checkpoint while continuing to use the same producer.

This implements the Python SDK portion of s2-streamstore/s2#666.

Validation

  • uv run --group test pytest tests/test_producer.py -q — 4 passed
  • non-integration test suite — 82 passed, 166 deselected
  • Ruff lint and format checks
  • mypy
  • Sphinx build with warnings treated as errors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant