fix: download media through the page instead of decrypting it again - #152
Open
DenysSamoiliuk wants to merge 1 commit into
Open
fix: download media through the page instead of decrypting it again#152DenysSamoiliuk wants to merge 1 commit into
DenysSamoiliuk wants to merge 1 commit into
Conversation
Attachment downloads fail on WhatsApp Web 2.3000.x in two separate ways, and both are invisible from node - the page hands back nothing but a minified class name. `Msg.get(serializedId)` misses, because the build indexes the collection under a key the serialized id no longer matches. whatsapp-web.js then falls back to `Msg.getMessagesById()`, whose IndexedDB lookup rejects the id with `DataError: Failed to execute 'get' on 'IDBObjectStore'`. That class is minified, so the error reaches the API as the opaque `t: t`. Resolve the message in the page instead - by serialized id, by the classic three part key, and finally by scanning the collection the chat already holds, matching on `id.id`, which no build has renamed. `downloadManager.downloadAndMaybeDecrypt` has to be fed `directPath`/`encFilehash`/`mediaKey`, and once the page has run a media retry those live on `msg.mediaObject`, not on the message. The stale key off `msg` decrypts to garbage and WhatsApp's own sniffer answers `InvalidMediaFileType: Unexpected mimetype application/octet-stream for media type image`, or `MediaDecryptionError: decryptMedia: hmac mismatch` when it gets further. `msg.downloadMedia()` already decrypts and parks the blob in WhatsApp's own cache, so take it from there and let the page own the crypto - the same move upstream made in wwebjs/whatsapp-web.js#201697, which is on main but not in any release yet. Wait for the blob rather than for a stage, and keep asking on every round: when several media arrive at once the first request goes nowhere and the stage never moves, so passively polling can only time out. `REUPLOADING` is waited out without asking again, since the page is already on it, and `mediaData` disappearing mid-download is treated as one more stage to sit through instead of a TypeError. Bounded by MEDIA_RESOLVE_TIMEOUT_MS, default 10s. Every failure now says which of these it was, in the logs and in the error the caller gets.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Attachment downloads fail on WhatsApp Web 2.3000.x. From node the failure is invisible — the page hands back nothing but a minified class name, so the API answers
t: t.There are two independent causes behind that.
1. The message is never found. The build indexes the
Msgcollection under a key the serialized id no longer matches, soMsg.get(serializedId)misses. whatsapp-web.js then falls back toMsg.getMessagesById(), whose IndexedDB lookup rejects the id:That class is minified, which is where the opaque
t: tcomes from.2. Re-decrypting uses stale key material.
downloadManager.downloadAndMaybeDecrypthas to be feddirectPath/encFilehash/mediaKey. Once the page has run a media retry, the fresh values live onmsg.mediaObject, not on the message — so the key read offmsgdecrypts to garbage and WhatsApp's own sniffer reports:The fix
Resolve the message in the page: by serialized id, then by the classic three-part key, then by scanning the collection the chat already holds, matching on
id.id— which no build has renamed. The IndexedDB fallback is never reached.Then let the page own the crypto.
msg.downloadMedia()already decrypts and parks the blob in WhatsApp's own cache, so read it fromInMemoryMediaBlobCache/mediaObject.mediaBlobinstead of decrypting a second time. This is the same move upstream made in wwebjs/whatsapp-web.js#201697, which is onmainbut not in any release yet.Waiting is keyed on the blob being there, not on
mediaStage:RESOLVEDbehind with nothing to read, so the stage cannot gate the callREUPLOADINGis waited out without asking again, since the page is already re-uploadingmediaDatadisappearing mid-download is one more stage to sit through, not aTypeErrorBounded by a new
MEDIA_RESOLVE_TIMEOUT_MS(default 10s). Every failure now names its own cause, in the log and in the error the caller gets, instead of a minified class name.Testing
tests/mediaDownload.test.js— 16 cases against a fake page that behaves like the broken build: each resolution path, the IndexedDB fallback never being reached, re-decryption never being reached, cache eviction, a dropped request,REUPLOADING,mediaDatavanishing, and each failure reason.Running in production on a busy line since the fix landed: 169 of 169 attachments failed in 24h before, zero after.