Repository navigation
feat(dwctl): accept gzip, zstd and br request bodies and compressed batch files (DW-985) - #2082
Conversation
…atch files (DW-985) Compressed bodies used to fail with misleading errors: a gzipped chat request was reported as missing its model, and a .jsonl.gz batch file as invalid UTF-8 on line 1. Batch files repeat the same prompt on every line and shrink 10-30x compressed, so accepting them saves customers most of their upload time. The inference body limit now decodes gzip, zstd and br before any other layer reads the body, applies the limit to the decoded size and refuses other codings with a 415 that lists the accepted ones. File uploads recognise gzip and zstd by their magic numbers and decode them as they stream, so the size limit, parser and stored file see decoded content.
Deploying control-layer with
|
| Latest commit: |
e6be2db
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://aca70532.control-layer.pages.dev |
| Branch Preview URL: | https://steffen-dw-985-accept-compre.control-layer.pages.dev |
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.
|
Running ultrareview automatically — this PR appears to change authentication, authorization, or input validation, where a missed bug can expose data or grant the wrong access, so a deeper multi-pass review is worth running. I'll post findings when complete. |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
Ultrareview ran automatically — this PR appears to change authentication, authorization, or input validation, where a missed bug can expose data or grant the wrong access, so a deeper multi-pass review is worth running.
All reported issues were addressed across 9 files
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
…(DW-985) brotli::Decompressor accepts the large-window extension, whose window of up to 1 GiB is allocated before any output, so a tiny body could exhaust memory past the output bound. Decode with the strict state, which keeps RFC 7932's 16 MiB ceiling. async-compression's zstd decoder stopped after the first frame, so a file of concatenated frames was stored truncated. Read them all, and recognise a stream that opens with a skippable frame as zstd.
There was a problem hiding this comment.
All reported issues were addressed across 4 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
Every other codec reads at most max_bytes + 1 decoded bytes; brotli could buffer a further 64 KiB chunk before checking the limit.
There was a problem hiding this comment.
0 issues found across 1 file (changes from recent commits).
Requires human review: Adds gzip/zstd/br decoding for /ai/v1 inference bodies and gzip/zstd batch uploads in dwctl, with new size limits and 413/415/400 handling. This is a large production request-path feature, so it needs an owner's sign-off.
View guided diff | Turn on auto-fix | Re-trigger cubic
… decodes (DW-985) Before compression was supported the Content-Encoding header was ignored, so a plain JSON body under a wrong or meaningless coding (a proxy's stray gzip, a client's utf-8) worked. Refusing it now would break those clients for nothing: pass it on as plain and count it instead. Decoding runs before authentication, and a few kilobytes of compressed body can expand to the body limit plus a decoder window. Bound decodes to one per CPU; requests over the bound wait rather than fail, so legitimate traffic is only delayed and uncompressed bodies never queue.
There was a problem hiding this comment.
All reported issues were addressed across 2 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
… (DW-985) A request dropped mid-decode (the client went away) released its slot while the blocking decode ran on, so disconnecting clients could exceed the one-per-CPU bound. Move the permit into the blocking task.
There was a problem hiding this comment.
0 issues found across 1 file (changes from recent commits).
Requires human review: Auto-approval blocked by 1 unresolved issue from previous reviews.
View guided diff | Turn on auto-fix | Re-trigger cubic
…pes (DW-985) (#2083) Stacked on #2082. Completes DW-985. ## Why RFC 9110 §15.5.16 reserves 415 for a request whose content format or coding the server doesn't support. #2082 returns it for unsupported content codings. Three other refusals in the same class still came back as a 400, which tells the client its request was malformed rather than sent in the wrong format: - **`POST /ai/v1/files` without `multipart/form-data`,** e.g. a raw JSONL body. This was a 400 "Invalid multipart request: invalid boundary", and it took an upload slot before being refused. - **A batch file compressed with xz, bzip2, zip or lz4.** #2082 names the format but returned a 400. - **Strict-mode `/responses` with a body not declared as JSON.** This was a 400 `invalid_content_type`. All three already fail today, so changing the status breaks no working client. ## What changes - **dwctl:** a new `Error::UnsupportedMediaType` variant, returned as a 415. - **File uploads:** a request without a `multipart/form-data` content type gets a 415 before it takes an upload slot. A multipart request with a broken form, such as a bad boundary, is still a 400. Files compressed in a format we don't decode also get a 415. - **onwards, strict `/responses`:** a missing JSON content type returns 415. The code stays `invalid_content_type`, so `onwards_rejections_total` series keep their label. - **New metric:** `dwctl_inference_request_content_types_total{content_type}` counts the content type declared on each POST to the inference endpoints, i.e. everything under `/ai/v1/*` except the file and batch routes, which already check theirs. Values are grouped into json, form, multipart, text, other and none. `/chat/completions`, `/completions` and `/embeddings` parse the body whatever it declares, and `curl -d` without `-H` declares a form body. Rejecting on content type there could break working clients, so this measures how many such requests there are before anyone decides. ## Not in this PR - **Rejecting non-JSON content types on the lenient endpoints:** a follow-up, once the new metric shows how much traffic it would affect. - **Non-UTF-8 `charset` parameters:** deliberately left alone for the same reason. ## Testing - `test_upload_that_is_not_a_multipart_form_returns_415`: a JSONL body, a form body and a missing content type each get a 415. A multipart body with a broken form still gets a 400. - `test_upload_broken_or_unsupported_compression_names_the_problem`: xz gets a 415, and truncated gzip still gets a 400. - `responses_bodies_not_sent_as_json_are_counted_as_such` in onwards: now asserts a 415 and the unchanged rejection code. - `classifies_declared_content_types`: covers the metric's labels. Co-authored-by: Steffen Ridderbusch <steffen@doubleword.ai>
Closes part of DW-985 (compression; the remaining 415s follow in a stacked PR).
Why
Clients can't send us compressed bodies today, and the ones that try get a misleading error. A gzipped
/ai/v1/chat/completionsbody reached the model lookup as raw bytes and came back as a 400 saying themodelparameter was missing. A.jsonl.gzbatch file failed as "invalid UTF-8 on line 1".Batch files gain the most. Every line repeats the same system prompt and schema, so a batch file compresses about twice as well with zstd or brotli as with gzip (roughly 23–29× against gzip's 12× on a synthetic file). A single chat request compresses about 3–4× whichever codec is used.
What changes
Realtime inference bodies (
/ai/v1/*).limit_inference_bodynow decodesContent-Encoding: gzip(andx-gzip),zstdandbrbefore any other inference layer reads the body. Inference routing, request logging, translation, the cache and onwards all keep seeing plain JSON, and upstreams receive uncompressed bodies:Content-Encodingis removed andContent-Lengthis set to the decoded size.limits.requests.max_body_sizeapplies to the decoded size, since that is what every inner layer buffers. Decoding stops as soon as the output passes the limit, so a small zip bomb costs at most the limit, and it gets a 413.zstdcontent-coding encoders from exceeding. A frame that declares a decoded size over the limit gets a 413 straight away.gzip, br) gets a 415. The response includesAccept-Encoding: gzip, zstd, br(RFC 9110 §15.5.16) and codeunsupported_content_encoding.body_decode_failed.gzipor a client'sutf-8. Such a body is now passed on as plain and counted indwctl_inference_bodies_mislabelled_total{declared}, instead of getting the 415 or 400 above. It only takes this path when the body fails to decode, so a real compressed body never does./messages, OpenAI's otherwise.onwards_rejections_totallike the other layers' refusals. Successful decodes are counted indwctl_inference_bodies_decoded_total{encoding}.Batch file uploads (
POST /ai/v1/files). A file part that starts with a gzip or zstd magic number is decoded as it streams in, with async-compression's tokio decoders.max_file_size, the JSONL parser and the stored file all see the decoded content. The reportedbytesis the decoded size, andbatch.jsonl.gzis stored asbatch.jsonl.zstdCLI decompresses without extra flags also uploads, including--longfiles. That window is allocated per upload, but uploads are capped bymax_concurrent_uploads. Tighten it if that's too much memory for your deployment.Formats are detected from the content, not from the file name or the part's content type, which are often missing or wrong. No JSONL file can start with any of these signatures.
Not in this PR
Testing
inference::body_limit: each coding round-trips, at the limit and with limits disabled. Unknown and layered codings get a 415. Plain JSON under any wrong coding passes through (passes_plain_json_under_a_wrong_content_encoding_through, and end to end in both modes). A decode waits while every slot is taken (decodes_wait_for_a_free_slot). Zip bombs get a 413 for all three codecs. Oversized zstd windows are refused. Corrupt data gets a 400./messagesgets the Anthropic shape. Refusals are counted.upload_compression: gzip (including multi-member) and zstd round-trip whatever the chunking. Truncated and corrupt files report a decode error, and upload errors pass through untouched, so length-limit detection still works. Covers format sniffing and filename stripping.files: compressed uploads end to end (stored name, decoded size, download content), and broken and unsupported compression name the problem.