Skip to content

fix(download): save every download byte for byte whatever its Content-Type - #5

Merged
ding113 merged 1 commit into
mainfrom
fix/raw-download-bytes
Oct 18, 2026
Merged

ding113 merged 1 commit into
mainfrom
fix/raw-download-bytes

Conversation

@ding113

@ding113 ding113 commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

Problem

cchp requests download-part <request-id> 'client%2Fresponse.frames' --output out.sse returned ok: true, file.bytes: 0, content_type: text/event-stream, and out.sse was empty, while curl with the same token and URL received the whole part. The same happened for attempt%2F<n>%2Fresponse.frames. JSON parts such as client%2Frequest.body worked.

Downloads ran through Restish's response formatter. For a text/event-stream (or NDJSON) response Restish takes its stream path and hands each event to the formatter (FormatValue); cchp's capture formatter drops events for downloads, so res.Raw stayed empty and saveDownload wrote 0 bytes and reported success.

Change

  • Downloads ask Restish for the raw body: no output format, so Restish writes the body to its non-terminal stdout byte for byte for every Content-Type (stream types included, content encodings decoded). The process HTTP transport, which already guards every request, now also records the head of each response; the engine builds the result from that head and the raw bytes.
  • cchp-gen classifies a response whose schema is a binary string as a download whatever its media types. CCHP serves an archive part with the part's own type (application/json, text/event-stream, application/x-ndjson, application/octet-stream).
  • operation-policy.yaml gains listing: for getUsageLogArchiveRaw it names getUsageLogArchive. When a download arrives empty, cchp reads the listing; if the listing shows the part as not empty the command fails with server_error (exit 10), never ok: true.
  • Error responses keep failing as before (no file is written), and --output - writes the raw bytes to stdout.
  • downloadRequestArchiveBundle (binary schema under application/json) becomes a download as well and needs --output.

Tests

internal/runner/download_test.go runs the real runner and engine against an HTTP server that behaves like CCHP's archive endpoints, and compares each saved file with what curl receives:

Case Content-Type Checks
JSON body application/json length and sha256 equal curl's; reported bytes, sha256 and content type
SSE client frames (4000 events) text/event-stream same
SSE attempt frames text/event-stream; charset=utf-8 same
octet-stream with NUL and CR bytes application/octet-stream same
NDJSON / JSONL application/x-ndjson, application/jsonl same
keepalive-only SSE, plain text text/event-stream, text/plain same
--output - text/event-stream stdout bytes equal the body
genuinely empty part application/json empty file, ok, sha256 of empty input
empty body of a listed 345678-byte part text/event-stream fails with server_error, exit 10
503 problem response application/problem+json fails with the problem and request id, no file left

cmd/cchp-gen/response_test.go covers the response classification.

Also run against a real CCHP gateway (feature branch of claude-code-hub-plus) with a stored request archive of 10 parts: v0.1.1 saved all 6 response.frames parts as empty files with ok: true; this branch saves all 10 parts with the same length and sha256 as curl.

go test -race ./..., go vet ./..., go vet -tags e2e ./..., golangci-lint run ./... and go run ./cmd/cchp-gen -check pass locally.

🤖 Generated with Claude Code

…-Type

Downloads ran through Restish's response formatter. When the server sent a
request archive part as text/event-stream or NDJSON, Restish decoded it as a
stream and handed the events to the capture formatter, which drops them for
downloads, so cchp saved an empty file and reported ok: true.

Downloads now ask Restish for the raw body (no output format), which it writes
byte for byte for every Content-Type; the process transport records the
response head. A response whose schema is a binary string is a download
whatever its media types. An empty download of a part that the request
archive lists as not empty fails with server_error.

Tests download JSON, SSE client and attempt frames, octet-stream, NDJSON,
JSONL and plain-text parts, a genuinely empty part and an error response from
a server that behaves like CCHP, and compare each saved file's length and
sha256 with what curl receives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-18T16:44:01.699484Z 8afae8c PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8afae8c15d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/runner/runner.go
Comment on lines +231 to +232
req := engine.Request{Op: op, Retry: op.Policy.Retry, HeaderTimeout: o.Timeout, MaxEvents: o.MaxEvents, DryRun: o.DryRun,
RawBody: op.Response.Kind == catalog.ResponseBinary}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep result metadata off raw download stdout

When a caller uses --output -, this binary path writes the response bytes to a.io.Out, but runOperation subsequently calls emit; redirected stdout automatically enables agent mode, so contract.Write appends a JSON result to the downloaded bytes (and terminal mode can append status/request metadata). This affects the newly classified downloadRequestArchiveBundle as well as the raw archive-part behavior being fixed here, so the command's stdout is not the byte-for-byte file asserted by the new test; suppress result emission for this mode or send metadata to a different stream.

Useful? React with 👍 / 👎.

@ding113
ding113 merged commit c616f58 into main Oct 18, 2026
6 checks passed
ding113 added a commit that referenced this pull request Oct 21, 2026
Move the Unreleased download fixes (#5) into v0.1.2 and add the CCHP
v1.0.10 spec sync with its new, changed and removed commands (#6).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ding113 added a commit that referenced this pull request Oct 21, 2026
Sync spec/upstream/control-plane.yaml to CCHP v1.0.10 (b0bf159de) with
make sync-spec and regenerate the catalog, schemas, engine spec and
docs/coverage.md: 621 operations, 514 available, 107 excluded with reasons,
0 unsupported.

- 62 new operations get names, groups, English summaries and policies;
  nine are excluded with reasons.
- listProviders became listAdminProviders; cchp providers overview keeps
  its name.
- listModerationViolations and streamModerationViolations are removed with
  cchp safety violations and watch-violations.
- Newly paged lists get pagination policies and new parameters get English
  help.
- The e2e pagination test creates four providers.
- CHANGELOG: v0.1.2 with the Unreleased download fixes (#5) and this sync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant