fix: preserve additional top-level properties on AirbyteStateMessage

[devin-ai-integration]

Summary

Python-CDK destinations on platform 1.7+ fail with java.lang.IllegalStateException: State message does not contain id because the platform attaches a dynamic id to each source STATE message (as an additional property) and requires it on the state messages the destination echoes back — but AirbyteStateMessage is a plain dataclass serialized with serpyco-rs, which silently drops unknown top-level keys on a load/dump round trip. AirbyteStateBlob already preserves extras via a CustomType; AirbyteStateMessage did not.

This adds AirbyteStateMessageType(CustomType[AirbyteStateMessage, Dict[str, Any]]) in airbyte_protocol_serializers.py:

• deserialize: loads known fields with an inner serializer (blob-only resolver to avoid recursion), then stores any unknown top-level keys on the instance as additional_properties.
• serialize: dumps known fields, then merges additional_properties back (never overwriting known keys).
• custom_type_resolver now resolves AirbyteStateMessage → AirbyteStateMessageType, so both AirbyteMessageSerializer and AirbyteStateMessageSerializer preserve extras.

The AirbyteStateMessage dataclass and its public fields are unchanged; when a message has no extra keys, output is byte-identical to before.

Regression tests (unit_tests/test_airbyte_protocol_serializers.py) assert a STATE message with an extra "id": 42 survives load→dump for STREAM state, GLOBAL state, and the direct state serializer, and that no additional_properties key leaks when there are no extras. All four fail without the fix (3 round-trip failures verified).

Context / prior art:

• destination-milvus 0.0.60→0.0.61 bumped airbyte-cdk 0.81.6 (pydantic, extra=allow, preserved id) to >=7.7.0 (serpyco dataclasses, drops id), breaking all syncs: https://github.com/airbytehq/oncall/issues/13391 (this affects every Python-CDK destination on CDK ≥6, e.g. milvus/pinecone/qdrant/weaviate/chroma/astra/typesense).
• fix(destination-typesense): preserve state message id field with CDK patch airbyte#70420 worked around the same bug in destination-typesense with a per-connector monkey patch; this is the CDK-level fix.
• Related: destination-typesense: State message does not contain id error with Airbyte 1.7+ airbyte#70421, SFTP-JSON connector - Receive same sync error with any source connector ("State message does not contain id") airbyte#68593, State message does not contain id - Connection state remains empty with custom destination connector airbyte#69138, Databases CDC not working with Python CDK Destination Connectors airbyte#55697.

Follow-up: once released, bump destination-milvus (and siblings) to a CDK version containing this fix.

Requested via /ai-fix on the oncall issue.

Test coverage

• poetry run pytest unit_tests/test_airbyte_protocol_serializers.py -q — 4 passed (fail without fix)
• poetry run pytest unit_tests/destinations -q — 74 passed
• poetry run ruff check ., poetry run ruff format --check ., poetry run mypy --config-file mypy.ini airbyte_cdk — clean

Link to Devin session: https://app.devin.ai/sessions/155df3ea04fd462faac82204f95c62d9

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1787726398-preserve-state-message-additional-properties#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1787726398-preserve-state-message-additional-properties

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

• /autofix - Fixes most formatting and linting issues
• /poetry-lock - Updates poetry.lock file
• /test - Runs connector tests with the updated CDK
• /prerelease - Triggers a prerelease publish with default arguments
• /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
• /poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

• CDK API Reference

📝 Edit this welcome message.

[Copilot]

Pull request overview

This PR fixes a protocol serialization bug in the Python CDK where unknown top-level keys on AirbyteStateMessage (notably the platform-attached dynamic id) were being silently dropped during a load → dump round trip, breaking destinations on platform 1.7+ that require the id to be echoed back.

Changes:

• Introduces a CustomType (AirbyteStateMessageType) to preserve and round-trip unknown top-level state-message properties via an additional_properties attribute.
• Updates the global custom_type_resolver so AirbyteStateMessage always uses the new custom type (including when nested under AirbyteMessage).
• Adds regression tests covering STREAM and GLOBAL state round-trips, plus direct state-message serializer behavior and the “no leakage when no extras” case.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
unit_tests/test_airbyte_protocol_serializers.py	Adds regression tests asserting unknown top-level properties (e.g., "id") survive load→dump for state messages.
airbyte_cdk/models/airbyte_protocol_serializers.py	Adds AirbyteStateMessageType and updates custom_type_resolver to preserve unknown top-level keys on AirbyteStateMessage.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

PyTest Results (Fast)

4 367 tests  +4   4 356 ✅ +4   10m 47s ⏱️ + 2m 0s
    1 suites ±0      11 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit ddee57a. ± Comparison against base commit 4855c2d.

[github-actions]

PyTest Results (Full)

4 370 tests  +4   4 358 ✅ +4   13m 26s ⏱️ -34s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit ddee57a. ± Comparison against base commit 4855c2d.