Skip to content

Raise httpc's header limit for archive downloads - #135

Merged
polvalente merged 1 commit into
elixir-nx:mainfrom
aptinio:raise-httpc-header-limit
Oct 24, 2026
Merged

polvalente merged 1 commit into
elixir-nx:mainfrom
aptinio:raise-httpc-header-limit

Conversation

@aptinio

@aptinio aptinio commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

GitHub's release-download redirects now send about 11 KB of response headers, about 9.2 KB of it a content-security-policy header. Erlang/OTP 29.0.6, 28.5.0.6 and 27.3.4.17 added a max_header_size request option to httpc that defaults to 10,240 bytes, so downloading the precompiled XLA archive fails on those releases and later with {:header_too_long, {:max, 10240}}.

This passes max_header_size: 65_536 with the request options in XLA.Utils.download/3, the same change as philss/rustler_precompiled#113 and elixir-lang/elixir_make#98.

Older releases don't know the option. The download still works there, but httpc logs Invalid option {max_header_size,65536} ignored once per request.

Checks:

  • Against a local server that answers with 12 KiB of headers, XLA.Utils.download/3 returns {:error, "reason: {:header_too_long, {:max, 10240}}"} on main with OTP 29.1.2 and {:ok, ...} with this change. On OTP 29.0.3, which predates the option, it succeeds with the notice above.
  • With the same line changed in xla 0.10.0, a cold exla compile on OTP 29.1.2 downloads the precompiled archive from GitHub releases.
  • mix format --check-formatted and mix compile --warnings-as-errors pass.

GitHub's release-download redirects now send about 11 KB of response
headers, most of it a content-security-policy. That exceeds the 10 KiB
max_header_size httpc enforces by default in recent Erlang/OTP patch
releases, so precompiled XLA archive downloads fail with header_too_long.
Raise the limit to 64 KiB for these requests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants