dedupe.py
Review the Python test suite next to the implementation. It maps out the specific edge cases and retry storms that trigger duplicate deliveries in production queues.
Idempotency is a hard requirement. When a consumer processes a message twice, the on-call engineer gets paged. This utility drops duplicate entries from a slice and preserves first-seen order.
The implementation relies entirely on the standard library. You do not need to provision background services or pull third-party packages to run it in your pipeline.