fix: restore Jackson 2.x property order in RqueueRedisSerializer to prevent stale processing-queue entries after 3.x → 4.x upgrade

[roxid]

Description

RQueue 4.x switched to Jackson 3.x, which defaults to alphabetical property ordering. RQueue 3.x used Jackson 2.x with declaration order. The same RqueueMessage therefore serialises to different bytes depending on which version enqueued it, causing byte-exact ZSCORE/ZREM lookups in parkForRetry and ack to silently miss — leaving 3.x messages stranded in the processing queue and re-delivered indefinitely via the visibility timeout.

Fixes #299

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• This change requires a documentation update

How Has This Been Tested?

• Integration Test: Inserts a genuine Jackson 2.x-serialized message directly into the processing ZSET via a passthrough template, then calls parkForRetry and ack using the live serializer. Using different serializers for write and lookup is intentional — if both sides used the same serializer the test would trivially pass regardless of whether the fix is present.

Test Configuration:

• Spring Version: 7.0.3
• Spring Boot Version: 4.0.1
• Redis Driver Version: Lettuce 7.2.1.RELEASE (embedded Redis 1.4.3)

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[sonus21]

Can you please bump the version in root to RC10?

[roxid]

Note that any 4.x deployment that has messages currently sitting in the processing queue should drain that queue before deploying this fix. move_expired_message.lua preserves raw bytes verbatim when rescuing messages, so alphabetical-order bytes written by an unpatched 4.x instance are never corrected by the rescue path — the ZSCORE miss will recur on every re-delivery, creating the same infinite loop as the original bug, just in reverse.

[sonus21]

Can we add this as a prop?

[roxid]

A property based solution seems to get rather messy. I'll try to implement a more robust solution (e.g. making the processing-queue lookups in move_message_zset.lua, move_message_list.lua, and ack using an ID-based fallback).

[sonus21]

A property based solution seems to get rather messy. I'll try to implement a more robust solution (e.g. making the processing-queue lookups in move_message_zset.lua, move_message_list.lua, and ack using an ID-based fallback).

This would be quick change right? We can add a property rqueue.serialization.property.order=ALPHABETICALLY|DEFAULT.

[roxid]

DEFAULT (declaration order, matching Jackson 2.x / RQueue 3.x) as the out-of-the-box value is intended? Meaning projects already using 4.x are required to explicitly set ALPHABETICALLY.

Maybe using ALPHABETICALLY as the default value, naming the old declaration order LEGACY and adding documentation to the migration guide is a more future proof approach here.

[sonus21]

DEFAULT (declaration order, matching Jackson 2.x / RQueue 3.x) as the out-of-the-box value is intended? Meaning projects already using 4.x are required to explicitly set ALPHABETICALLY.

Maybe using ALPHABETICALLY as the default value, naming the old declaration order LEGACY and adding documentation to the migration guide is a more future proof approach here.

Yeah this make sense, Also we can take a approach used by Lombok. Serialization format jackson2.x or jackson3+. Both of these would work, but adding legacy does not tell us why this is called legacy.

[roxid]

Both of these would work, but adding legacy does not tell us why this is called legacy.

You're right, naming should be consistent and describe the actual behaviour.

Added rqueue.serialization.property.order=ALPHABETICAL|DECLARATION to control JSON property ordering. ALPHABETICAL (Jackson 3.x native) is the default for new 4.x deployments. Set DECLARATION when upgrading from 3.x with messages still present in Redis queues.