Repository navigation
feat(plugin): implement chunked body fetching and error handling for … - #18
Merged
Merged
Conversation
…oversized responses
Validate announced body_size, reject overruns, build the body without a second copy, retry chunk reads on empty host replies, drop the doubled op prefix in host errors, and reuse errEmptyHostResponse in CallHostFunction.
This was referenced Nov 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A WASM plugin could not fetch or return anything larger than about 1 MiB, and when it hit that limit it failed silently:
Fetchreportedplugin: fetch: empty response from host, which reads like a network outage, and a plugin's own oversized result came back to the host as nothing at all.This PR removes both limits and, where a limit genuinely applies, turns the silent failure into an explicit, typed error. It is the SDK half of #16 (the host half lives in Paca core's plugin runtime).
Why
The SDK has one fixed 1 MiB arena (
mallocBuffer) that the host writes request data and host-function results into. Everything shared it:{"status","body","headers"}) had to fit in what was left of the arena. When it did not, the host had no room even for an error message and returned an empty response. A 1,168,331-byte GitHub issue page produced a 1,238,310-byte envelope and failed; the same code with a smaller page worked.packWASMResultcopied a plugin's response back into the same arena and returned0when it did not fit. For an HTTP-shaped response the body is base64-encoded, so a 902 KB provider result became a 1.2 MB response and was silently dropped. Core then reported the provider as "temporarily unavailable" (HTTP 503).FetchandGetBrandingreleased the arena after reading a result. DB, KV, cache and config reads never did, so earlier host calls in the same request ate the space a later one needed.Changes
Chunked fetch (
fetch_chunks.go,wasm_backends.go)Fetchnow sends"chunked": true. If the whole envelope fits, nothing changes (one round trip, same result). If not, the host parks the body and answers with the status, headers, abody_handleandbody_size; the SDK then reads the body in 256 KiB pieces (base64 in the JSON envelope, so it is binary-safe) and returns the completeFetchResponse.Body.Fetchkeeps its signature and result. Callers get large responses without code changes.Explicit errors (
host_limits.go)ErrResponseTooLarge, returned wrapped (errors.Is(err, plugin.ErrResponseTooLarge)), for a result that cannot be delivered.FetchResponsegains aCodefield carrying the host'sresponse_too_largeclassification.Large plugin results (
wasm_exports.go,wasm_result.go)packWASMResultnow returns the result from the guest heap instead of the arena. The host reads a result from whatever memory address it is given, so results are no longer bounded by the arena (this works with existing hosts too).Errorfield of a condition or action result) instead of an empty reply.Arena hygiene
Fetchalready did.HandleEventcopies the event payload, since a handler's host calls may now reset the arena whileEvent.Payloadstill pointed into it.Compatibility
chunkedis ignored; an oversized fetch fails with the clearer "host returned no data" message instead of the misleading one. Large plugin results already workchunkedflag, so an oversized fetch gets an explicitresponse_too_largeerror, not an empty responseNo wire-contract change to the Git provider SPI, and no new required host function.
Notes and limits
Tests
fetch_chunks_test.go: reassembling a body across many chunks, shrinking the chunk size when the arena is partly full, giving up at the minimum withErrResponseTooLarge, empty or short chunks not looping forever, and the wire shapes.host_limits_test.go: error classification and the new wording.wasm_result_test.go: results at the 16 MiB limit pass; one byte over is replaced by an explicit error.go vet,go test -race ./...andGOOS=wasip1 GOARCH=wasm go vet .all pass.