Skip to content

fix(async): don't crash streaming pushes on pooled Node Buffers (#278) - #292

Open
maximilliangrand wants to merge 1 commit into
101arrowz:masterfrom
maximilliangrand:fix/async-stream-pooled-buffer-transfer
Open

maximilliangrand wants to merge 1 commit into
101arrowz:masterfrom
maximilliangrand:fix/async-stream-pooled-buffer-transfer

Conversation

@maximilliangrand

Copy link
Copy Markdown

Fixes #278.

Repro (Node):

const { Zip, AsyncZipDeflate, unzipSync } = require('fflate');
const zd = new AsyncZipDeflate('a.bin', { level: 6 });
new Zip((e,d,f)=>{}).add(zd);
zd.push(Buffer.from('x'.repeat(2000)), true); // DataCloneError: Cannot transfer object of unsupported type

Cause: The async streaming classes transfer d.buffer on every push. A Node Buffer (from Buffer.from/readFileSync) is a view into the shared pool ArrayBuffer that Node marks untransferable, so the transfer throws. #227 (4b7a6cb) already filters untransferable buffers, but only on the worker's first message; streaming pushes call postMessage directly and bypass it.

Fix: Wrap the node worker's postMessage so the transfer list is filtered on every message; untransferable buffers fall back to a structured clone (the same graceful degradation already used for cross-realm buffers). Fresh, transferable Uint8Arrays still transfer as before.

Tests/types/docs: adds an async regression test (test/5-async.ts, previously empty) covering AsyncZipDeflate and AsyncDeflate with a pooled Buffer; tsc clean; no public API or doc change.

…rrowz#278)

The 101arrowz#227 fix filtered untransferable buffers only on the worker's first
message. Streaming pushes (AsyncDeflate/AsyncZipDeflate/etc.) call the
worker's postMessage directly, so pushing a Node Buffer -- whose backing
ArrayBuffer is the shared, untransferable pool -- still threw
`DataCloneError: Cannot transfer object of unsupported type`.

Wrap the worker's postMessage so the transfer list is filtered on every
message; untransferable buffers fall back to a structured clone. Adds an
async regression test covering AsyncZipDeflate and AsyncDeflate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DataCloneError when pushing a Uint8Array into AsyncZipDeflate

1 participant