Skip to content

fetch() UTF-8-encodes outgoing header byte strings, changing the bytes sent #7734

Open

Description

@rexxars

I ran into this while fixing Last-Event-ID encoding in eventsource.

The EventSource spec requires sending the event ID encoded as UTF-8. With Fetch, those bytes are represented as a byte string in the header value. workerd UTF-8-encodes that string again, so the server receives a different ID.

For example, this Worker sends the UTF-8 bytes for café:

export default {
  fetch() {
    return fetch('http://127.0.0.1:8788', {
      headers: {'Last-Event-ID': 'caf\xc3\xa9'},
    })
  },
}

Run it locally against this Node server to inspect the received bytes:

import {createServer} from 'node:http'

createServer((req, res) => {
  const value = req.headers['last-event-id'] ?? ''
  res.end(Buffer.from(value, 'latin1').toString('hex'))
}).listen(8788, '127.0.0.1')
Header bytes
Expected 636166c3a9
Actual 636166c383c2a9

The server decodes the actual bytes as café.

Every character in the supplied value is within U+0000–U+00FF. The Fetch Headers API uses ByteString, whose Web IDL conversion preserves those values without UTF-8 encoding.

Reproduced with workerd 1.20261013.1 using compatibility date 2026-10-13 - enabling pedantic_wpt produces the same result.

Passing the original Unicode string works around this in workerd, but requires runtime-specific handling in libraries using Fetch.

This seems related to #4792 and #6927. The case here concerns outgoing header bytes, including values that are already valid byte strings.

Could a compatibility flag enable spec-compliant handling for outgoing headers too? I understand changing the default could break existing Workers, but an opt-in would let libraries send the same bytes across runtimes.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions