Can this library be used with WICG Direct Sockets for a WebTransport server?

WICG Direct Sockets exposes TCPServerSocket , UDPSocket, and TCPSocket in an Isolated Web App, on Chromium-based browsers.

I've successfully created an HTTP and WebSocket server using only JavaScript shipped in the browser - other than the above interfaces exposed in an IWA.

Can this library be used with WICG Direct Sockets for a WebTransport (HTTP, HTTP/2, HTTP/3, h2c) server?

Currently, it is impossible as only node.js sockets are used, and the C++ part for http/3 needs to be compiled with wasm.
The same is true for the http/2 (and WebSocket server); here, only the node.js socket functions are used.
An extension to this use case is possible in principle, but it is not on my agenda as it does not overlap with my use cases.

I think it's possible to simply rewrite the Node.js API parts to use Web API's and use WASI-SDK to produce WASM from C++, or convert the C++ to AssemblyScript then the WASM asc produces.

Thanks.

No I do not think that it is so simple. For WebTransport over Http/2 or WebTransport you would need to rewrite large parts that node.js HTTP, http/2 servers provide.
For http/3, it may be simpler as only the socket part is used from node.js, but then you need to compile with emscripten and rewrite the interface part of the C++ part, which currently uses an API to interface with node.js to interface with emscripten; this is a larger task.

No I do not think that it is so simple. For WebTransport over Http/2 or WebTransport you would need to rewrite large parts that node.js HTTP, http/2 servers provide.

I've done that before, more than once. E.g., rewriting wbn to use Web Cryptography API and substituting Uint8Array, et al. for Buffer; https://github.com/guest271314/direct-sockets-http-ws-server?tab=readme-ov-file#buildrebuild-wbn-bundlejs-from-webbundle-pluginspackagesrollup-plugin-webbundlesrcindexts-with-bun. The code was originally written for Node.js exclusively. I modified the code to work using node, deno, and bun.

For http/3, it may be simpler as only the socket part is used from node.js, but then you need to compile with emscripten and rewrite the interface part of the C++ part, which currently uses an API to interface with node.js to interface with emscripten; this is a larger task.

I've compiled the same algorithm to WASM from C++, C, Rust using WASI-SDK or clang, JavaScript using Facebook's Static Hermes with WASI-SDK and/or Emscripten, Bytecode Alliance's Javy; and compiled node-http2 module that was merged into Node.js to a bundle that can be used with node, deno, or bun using bun build.

I generally make an effort to write runtime agnostic, and when possible language agnostic code that can be run in different environments.

Sockets are generally the same between Node.js, Deno, Bun, and the browser implementation in TCPServerSocket, TCPSocket, and UDPSocket.

Right now I'm trying to figure out what the headers look like. Then I can try to do a HTTP/1.1 to HTTP/2 Upgrade, first. Or dive in to QUIC over UDP. I looked at Deno's implementation, that works to an appreciable degree, though that also has Deno-specific traits https://gist.github.com/guest271314/c1974f24fff86fe4708f901651be8079.

Another avenue I'm working on is using Chrome's built-in full-duplex streaming over fetch() between a ServiceWorker and a WindowClient or Client. That is the only browser implementation of HTTP/2 full-duplex streaming in a browser where fetch() does fulfill to a Response before then response is completed from the remote server. E.g., using an extension to stream from any arbitrary Web site into the single connection HTTP/2 connection between a WindowClient and a ServiceWorker on Chromium-based browsers

addEventListener("fetch", async (e) => {
  console.log(e.request.url, e.request.destination, e.clientId);
  if (e.request.url.includes("stream")) {
    e.respondWith(
      new Response(
        e.request.body
          .pipeThrough(new TextDecoderStream())
          .pipeThrough(
            new TransformStream({
              async start() {
                // getClientState(e.clientId);
                if (globalThis.port === null) {
                  readable = new ReadableStream({
                    start: (_) => {
                      return controller = _;
                    },
                    cancel(reason) {
                      console.log(reason);
                    },
                  });
                  reader = readable.getReader();
                  port = chrome.runtime.connectNative(
                    chrome.runtime.getManifest().short_name,
                  );
                  port.onMessage.addListener((message) => {
                    controller.enqueue(encoder.encode(message));
                  });
                  port.onDisconnect.addListener((e) => {
                    console.log(e);
                    if (chrome.runtime.lastError) {
                      console.log(chrome.runtime.lastError);
                    }
                    controller.close();
                    port = readable = controller = null;
                  });
                }
              },
              async transform(value, c) {
                console.log(value);
                port.postMessage(value);
                const { value: message, done } = await reader.read();
                console.log(message, done);
                c.enqueue(message);
              },
              flush() {
                console.log("flush");
              },
            }),
          ),
      ),
    );
  }
});

In DevTools in an arbitrary Web page

async function nativeMessageStream(u, rs) {
  function removeFrame() {
    const frames = document.querySelectorAll("[src^=chrome]");
    frames.forEach((w) => {
      w.remove();
    });
    return document.querySelectorAll("[src^=chrome]").length;
  }

  async function generateIdForPath(path) {
    return [
      ...[
        ...new Uint8Array(
          await crypto.subtle.digest("SHA-256", new TextEncoder().encode(path)),
        ),
      ].map((u8) => u8.toString(16).padStart(2, "0")).join("").slice(0, 32),
    ].map((hex) => String.fromCharCode(parseInt(hex, 16) + "a".charCodeAt(0)))
      .join("");
  }

  const url = new URL(
    `chrome-extension://${await generateIdForPath(u)}/transferableStream.html`,
  );
  return new Promise((resolve) => {
    function handleMessage(e) {
      console.log(e);
      if (e.origin === url.origin) {
        if (e.data instanceof ReadableStream) {
          resolve({
            readable: e.data.pipeThrough(
              new TransformStream({
                transform(value, c) {
                  c.enqueue(value);
                },
                flush() {
                  removeEventListener("message", handleMessage);
                  console.log(removeFrame());
                },
              }),
            ),
            transferableWindow,
          });
        } else {
          e.source.postMessage(rs, "*", [rs]);
        }
      }
    }
    addEventListener("message", handleMessage);
    const transferableWindow = document.createElement("iframe");
    transferableWindow.style.display = "none";
    transferableWindow.name = location.href;
    transferableWindow.src = url.href;
    transferableWindow.addEventListener("load", (e) => {
      console.log(e);
    });
    document.body.appendChild(transferableWindow);
  }).catch((err) => {
    throw err;
  });
}

var { readable: r, writable: w } = new TransformStream();
var writer = w.getWriter();
var encoder = new TextEncoder();
var { readable, transferableWindow } = await nativeMessageStream(
  "/home/user/half_duplex_stream",
  r,
);

Thereafter

await writer.write(encoder.encode("Streaming request");

which is enqued into the upload stream, and the response is sent back to the WindowClient in a single connection. I have kept ServiceWorkers active for over 48 hours (technically the connection can be indefinite https://plnkr.co/edit/bK1BfoSgjFUDwkIV?preview, and full-duplex, just using WHATWG Fetch) on a single HTTP/2 connection using this approach https://github.com/guest271314/persistent-serviceworker/tree/main/readablestream-fetch-respondwith.

and here for http/2 (and webtransport over websocket for the server side):
https://github.com/fails-components/webtransport/blob/master/main/lib/http2/node/client.js
https://github.com/fails-components/webtransport/blob/master/main/lib/http2/node/server.js

And please note, that currently browser support as client only Webtransport over http/3 and via the polyfill of the lib Webtransport over websocket.

I'll see what kind of damage I can do. I already re-wrote somebody elses Node.js WebSocket work to runtime agnostic JavaScript. Thanks.

@martenrichter

@martenrichter I have a question about WebSocket if you have the time and it's appropriate to ask here. Thanks.

Sure.

I'm trying to figure out why the WebSocket server code I adapted (https://github.com/guest271314/direct-sockets-http-ws-server/blob/main/assets/script.js#L1-L193, https://github.com/guest271314/direct-sockets-http-ws-server/blob/main/assets/script.js#L262-L291) from here https://gist.github.com/robertrypula/b813ffe23a9489bae1b677f1608676c8 stops sending back messages after around the second message.

E.g., something like this should work with the above implementation. Instead, only two (2) messages are sent back to the browser

var wss = new WebSocketStream("ws://0.0.0.0:44818");
console.log(wss);
wss.closed.catch((e)=>{console.log(e)});
var {readable, writable, } = await wss.opened.catch(console.error);
var writer = writable.getWriter();
var abortable = new AbortController();
var {signal, } = abortable;
// .pipeThrough(new TextDecoderStream())
var pipe = readable.pipeTo(new WritableStream({
  start(c) {
    console.log("Start", c);
  },
  async write(v) {
    console.log(v);
  },
  close() {
    console.log("Socket closed");
  },
  abort(reason) {
    console.log({ reason });
  },
}), {
  signal,
}, ).then(()=>console.log("Done")).catch((e)=>console.log(e));

var encoder = new TextEncoder();
var decoder = new TextDecoder();
var encode = (text)=>encoder.encode(text);
for (let i = 0; i < 10; i++) {
  await writer.write(encode("X".repeat(4096)));
  await writer.ready;
}

The real test: Do what I can do using WebTransport with echoing 7 MB

var data = new Uint8Array(1024**2*7);
for (let i = 0; i < data.length; i += 1024*4) {
  await writer.write(data.subarray(i, i + 1024*4));
  await writer.ready;
}

That works using Deno's builtin WebSocket server. So there must be an issue with the implementation I adapted for use in the browser.

I'm pretty sure this is the totality of the runtime agnostic implementation https://gist.github.com/guest271314/735377527389f1de6145f0ac71ca1e86.

It's hard to say from the code. I would probably add a lot of console.log to the server code and frame code.
What looks suspicious is that the remaingBytes are only used for the one chunk of data coming from the socket. So I think you are discarding data and losing sync in the websocket frames. But I may have missed something.

I built with clang just to compile those 3 .proto files, removed all references to protobuf and protoc in the CMakeList.txt, then just copied the files over in the build.

Then encountered this building for WASM

[ 34%] Building CXX object third_party/boringssl/src/CMakeFiles/crypto.dir/crypto/bio/connect.cc.obj
/media/user/1/webtransport/transports/http3-quiche/third_party/boringssl/src/crypto/bio/connect.cc:181:48: error: use of undeclared identifier 'SO_KEEPALIVE'
  181 |         ret = setsockopt(bio->num, SOL_SOCKET, SO_KEEPALIVE, (char *)&i,
      |                                                ^
/media/user/1/webtransport/transports/http3-quiche/third_party/boringssl/src/crypto/bio/connect.cc:191:15: error: use of undeclared identifier 'connect'
  191 |         ret = connect(bio->num, (struct sockaddr *)&c->them, c->them_length);
      |               ^
2 errors generated.
make[2]: *** [third_party/boringssl/src/CMakeFiles/crypto.dir/build.make:513: third_party/boringssl/src/CMakeFiles/crypto.dir/crypto/bio/connect.cc.obj] Error 1
make[1]: *** [CMakeFiles/Makefile2:5614: third_party/boringssl/src/CMakeFiles/crypto.dir/all] Error 2
make: *** [Makefile:156: all] Error 2

It might just be too much ground to cover to get this into WASM when it didn't start out with the idea to get into WASM. I'll keep trying, anyway.

Why wt? they are more of a hassle than they're worth; websockets get you very far.

If you're familiar with Android you can use the Chrome networking stack with via java @ cronet org.chromium.net:cronet-api:130.6723.31

So I started here:
https://github.com/GoogleChromeLabs/telnet-client/blob/main/src/echo.ts
https://github.com/GoogleChrome/samples/tree/gh-pages/webtransport
https://github.com/chromeos/iwa-sink/tree/main

feross got me thinking:
https://github.com/feross/chrome-net

but are you not able to get quinn wasm wired up? The web assembly spec stuff shifting so much can make it hard depending on what you need i.e., JSPI, mem64, tailcalls, threading etc, but you can see another iteration of me using it here (h3 tho not wt) ->

https://github.com/maceip/freehold/blob/main/platforms/web/isolated-web-app/assets/script.js

Where are you guys getting stuck?

@maceip

@maceip

Why wt?

Because it's supposedly possible

On Tue, Mar 11, 2025 at 12:35 AM Reilly Grant reillyg@chromium.org wrote:

That issue is obsolete. There is nothing stopping you from implementing a WebTransport server other than building an implementation using JavaScript or WebAssembly.
Reilly Grant | Software Engineer | reillyg@chromium.org | Google Chrome

On Mon, Mar 10, 2025 at 5:22 PM guest271314 guest271314@gmail.com wrote:

The goal with HTTPS/TLS being able to create a WebTransport server
with TCPSeverSocket, for full-duplex streaming using WHATWG Streams.

On Mon, Mar 10, 2025 at 11:53 PM Reilly Grant reillyg@chromium.org wrote:

There's nothing stopping you from implementing your own TLS client/server library on top of the Direct Sockets API. The API proposed here makes that easier by allowing you to validate certificates using the browser's certificate store instead of providing your own.
Reilly Grant | Software Engineer | reillyg@chromium.org | Google Chrome

hasn't been done, yet, and is sufficiently challenging to keep me interested,

but are you not able to get quinn wasm wired up?

No; not yet.

In pertinent part private-octopus/picoquic#2036 (reply in thread)

@huitema

I would be interested to see the results if you try that.

Alright, I'll try then. So far I've tried

Compile fails-components/webtransport (depends on quiche compiled to a .node file) Can this library be used with WICG Direct Sockets for a WebTransport server? fails-components/webtransport#354 which consistently got stuck on protobuf usages and then after then still have to deal with the .node file

Rust's quinn compiled to WASM, following up on the work by @maceip https://github.com/guest271314/quinn-wasm/tree/direct-sockets

Asked about compiling BiagioFesta/wtransport to WASM Can the server be compiled to JavaScript for use in Deno? BiagioFesta/wtransport#40

Asked about compiling quiche to WASM Has anybody compiled quiche to WASM? google/quiche#86

quico https://github.com/colocohen/quico (Node.js which I plan(ned) to convert to Web API's)

Currently days upon days of trying to get Google's Gemini to implement a WebTransport server in JavaScript https://gemini.google.com/share/a34176ee9792, which not infrequently freezes the browser tab; I sense I'm close, just need to complete the TLS handshake, I think

@maceip

@maceip There's no quinn_wasm_direct_sockets.js file in the linked repository, that I can see.

ah i remember now, its generated here
https://github.com/maceip/freehold/blob/7727fb939edb79c653777b77366f45a40dbce629/.github/workflows/build-quinn-wasm.yml#L235

but that is just a stub, you will need ->

https://github.com/bjorn3/browser_wasi_shim
1.5. compile quinn for wasi targeting wasm32-wasip1 (or P2).
implement Sockets in JS: browser_wasi_shim allows you to define "host functions" in js. You would implement the WASI socket syscalls (like sock_send, sock_recv) by mapping them to the browser's UDPSocket (DirectSockets API).
when quinn calls socket.send(), it triggers a WASI syscall, which browser_wasi_shim catches and passes to your JS implementation of UDPSocket.

...libp2p also has a good wt stack; https://github.com/achingbrain/webtransport-stop-sending

@maceip

@maceip Yeah, this https://github.com/guest271314/quinn-wasm/tree/direct-sockets doesn't work as described in The README, nor after I got the IWA and Direct Sockets parts working, yet.

@maceip

@maceip Google Gemini's WebTransport JavaScript Web API implementation - that doesn't get past the handshake

google-gemini-wt-webapi-impl.zip

@martenrichter

@martenrichter

Got something going here... https://github.com/guest271314/quic-zig-wasm/blob/wasm-experiment/wasm/README.md

@martenrichter

@martenrichter @sbc100

FWIW I got this TLS+QUIC+WebTransport sans-io state machine compiled to wasm32 from Zig source code to work as a WebTransport server in Chromium browser using Direct Sockets UDPSocket, and using Node.js, Deno, Bun, and txiki.js https://github.com/guest271314/webtransport/blob/runtime-agnostic/webtransport-example.js.

Ok, good. But it does not has much todo with this project here.

Ok, good. But it does not has much todo with this project here.

Yeah, was just letting you know the thing got done. Without quiche. I'm still trying to compile that to WASM. Still caught up on protobuf stuff.

We can have WebTransport in Node.js - and indeed across JavaScript runtimes -without quiche, and without Node API specific stuff.

@martenrichter

@martenrichter

Re w3c/webtransport#511

Well there is massive progress on node quic... @jasnell really did a great job.
In principle, one could implement WT with a manual implementation of http/3 and capsules.
For a native implementation, nghttp3 is the blocker as it waits for an RFC release of Webtransport over http/3 (see ngtcp2/nghttp3#442 and ngtcp2/nghttp3#501 ). Though one could backport it https://github.com/martenrichter/node/tree/nghttp3-webtransport-backport , the changes to get than Webtransport running are not big.

Any idea, when the RFC is to be released?

Here's an implementation written in Zig and compiled to WASM - and transpiled from WASM to JavaScript using wasm2lang that works using node, deno, bun, tjs, and in Chromium based browsers over Direct Sockets UDPSocket.

W3C banned me. If you could be so kind as to share the above over in that WebTransport issue that would be great.

No I will not. They are looking for a way for automated testing and exploring concepts. This is out of scope.

No I will not. They are looking for a way for automated testing and exploring concepts. This is out of scope.

Yeah, nothing is stopping you from testing that code in an automated way.

What a pity you won't share working code, for whatever reason...

Navigation Menu

Repository navigation

Can this library be used with WICG Direct Sockets for a WebTransport server? #354

87 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Can this library be used with WICG Direct Sockets for a WebTransport server? #354

Description

Activity

martenrichter commented on Mar 17, 2025

guest271314 commented on Mar 17, 2025

martenrichter commented on Mar 17, 2025

guest271314 commented on Mar 17, 2025

guest271314 commented on Mar 17, 2025

martenrichter commented on Mar 17, 2025

martenrichter commented on Mar 17, 2025

guest271314 commented on Mar 17, 2025

guest271314 commented on Apr 19, 2025

martenrichter commented on Apr 19, 2025

guest271314 commented on Apr 19, 2025

guest271314 commented on Apr 19, 2025

guest271314 commented on Apr 19, 2025

martenrichter commented on Apr 19, 2025

87 remaining items

guest271314 commented on Nov 26, 2025

maceip commented on Feb 21, 2026

guest271314 commented on Feb 21, 2026

guest271314 commented on Feb 21, 2026

maceip commented on Feb 21, 2026

guest271314 commented on Feb 21, 2026

guest271314 commented on Feb 21, 2026

guest271314 commented on Mar 22, 2026

guest271314 commented on Apr 12, 2026

martenrichter commented on Apr 12, 2026

guest271314 commented on Apr 12, 2026

guest271314 commented on Apr 12, 2026

guest271314 commented on May 24, 2026

martenrichter commented on May 24, 2026

guest271314 commented on May 24, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions