Add persistent IMAP UIDs, OAuth2 support, smart body verification, and checkpoint resumption

[tholden]

Summary of Improvements

This PR introduces several robustness, authentication, and safety enhancements to IMAPdedup:

1. Persistent IMAP UIDs (RFC 3501)

   • Replaces transient sequence numbers with immutable IMAP UIDs (server.uid(...)), ensuring operations are unaffected by new incoming emails or shifting index numbers.

2. Smart Body & Attachment Verification (--check-body\ / --no-check-body)

   • Enabled by default: when header checksums match, decodes plain text, normalizes line endings and whitespace, and compares SHA-256 cryptographic hashes of all binary attachments before flagging a duplicate.
   • Prevents false-positives caused by automated servers rewriting non-semantic HTML wrapper tags.

3. OAuth2 Modern Authentication (-o / --oauth2\ & --token-file)

   • Adds XOAUTH2 support for Office 365, Google Workspace, and other OAuth2 providers.
   • Includes automatic token file discovery and seamless in-flight token refresh.

4. Resilience, Checkpointing & Auto-Resumption

   • Implements chunk checkpointing so that any network interruption, timeout, or rate limit automatically reconnects and resumes from the last processed UID rather than restarting from message 1.
   • Automatically detects Microsoft Exchange rate-limiting (\Suggested Backoff Time), backs off, and resumes cleanly.
   • Increases socket timeouts to 300s to support mailboxes with hundreds of thousands of emails.

5. Custom Domain Server Mapping (--server-map)

   • Enables mapping custom hosted domains to IMAP server hostnames at runtime.

6. Enhanced Header Checksum (-c\ & -m)

   • Adds \Content-Type\ to the multi-header checksum alongside \From, \To, \Subject, \Date, \Cc, \Bcc, and \Message-ID.

7. Clean Piping & Pipeline Support (-l)

   • Directs status messages to \stderr\ during list mode so \stdout\ output is pure and clean for piping into \xargs\ or shell scripts.

8. Documentation

   • Fully updated \README.md\ detailing all new options, usage examples, and architecture improvements.

[quentinsf]

Thanks Tom, but that's too many changes for a single pull request. (It looks possibly AI-generated?)
Could you submit them individually with a little explanation? Thanks!

[tholden]

Almost entirely AI of course. Wouldn't it be weirder not to use AI these days?

The changes seem quite tightly linked: "Smart Body & Attachment Verification (--check-body) and Checkpointing are deeply intertwined with the UID refactoring (they rely on immutable UIDs rather than sequence numbers)."

So it would be a sequence of at least 4 PRs to be merged sequentially, and the end result would be this anyway.

I made these changes to make imapdedup usable for me, but they're here under free disposal.

(I would suggest you switch to UIDs in any case, as the current system is a recipe for deleting the wrong thing.)

[quentinsf]

OK, thanks, Tom - I'll try to get a chance to look through all of these!

(It's not that I'm not grateful for your contributions, but I have to be a bit careful about things like backward compatibility. There are some people who have been running this as part of an automation against unusual IMAP servers for many years, so I wouldn't want to break anything for them. Or at least, only break things one at a time!)