Skip to content

feat(quic): authenticate peers with certificates from a private CA - #154

Draft
ryanolson wants to merge 3 commits into
mainfrom
ai/quic-mtls
Draft

ryanolson wants to merge 3 commits into
mainfrom
ai/quic-mtls

Conversation

@ryanolson

@ryanolson ryanolson commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

What

The QUIC transport can authenticate both peers with certificates from a private CA (ACM Private CA, SPIRE, cert-manager), and the application decides on each connection before any frame on it is dispatched. The pinned self-signed certificate stays the default. Nothing changes for a caller that does not call QuicTransportBuilder::tls.

  • QuicTls::new(roots, chain, key) and QuicTransportBuilder::tls. Both ends present a chain and verify the other against the roots in the handshake. The listener requires a client certificate. The dialer verifies the server chain and the serverAuth usage, and not a name, because every dialer presents the name velo.
  • QuicTls::authorizer takes a QuicAuthorizer or a closure Fn(&PeerIdentity, &ConnectionOrigin) -> bool. PeerIdentity gives the leaf, the chain, and the URI SANs (spiffe://<deployment>/<class>/<cell>). ConnectionOrigin is Accepted { remote } or Dialed { peer, lane, remote }.
  • Where the decision is made:
    • Listener: in serve_connection, after the handshake and before the first accept_bi. Streams that the peer opened wait unaccepted in quinn, so nothing reaches route_frame.
    • Dialer: in the connection writer, after the handshake and before open_bi, the dialed reader, and the first write. Queued frames fail through their error handlers.
    • Health check: a dial too. A refused peer reports ConnectionFailed.
    • Every lane is its own connection, so every lane is decided on. A refused connection is closed with the code REFUSED. A panic in the authorizer is a refusal.
  • Rotation: QuicTls::rotate(chain, key) and QuicTls::set_roots(roots). The certificate resolver and both verifiers read the current values on each handshake, so new connections use them, also to peers that were registered before. QuicTransport::tls returns the handle.
  • Session resumption and 0-RTT are off in this mode. A test caught that a resumed TLS 1.3 session presents the identity of the first session after a rotation, and it skips certificate verification, so it would also outlive set_roots.
  • In CA mode the dialer's lane offset is random. Before, it came from the self-signed fingerprint.
  • New dependency: rustls-webpki >= 0.103.8 (already in the tree through rustls), to read URI SANs. rustls now needs >= 0.23.23. velo::transports::quic re-exports CertificateDer and PrivateKeyDer from rustls-pki-types.

Limits

  • A refusal by the listener can lose frames without a report. In TLS 1.3 the client finishes its handshake, and can write, before the server verifies the client certificate. The dialer logs a warning when the peer closes with REFUSED. A refusal by the dialer, and a server chain that the dialer does not trust, fail the queued frames through their error handlers.
  • All peers must use the same mode. A default transport has no client certificate.
  • The identity is given to the authorizer per connection. It is not attached to each inbound message or handler context.
  • TCP with TLS is out of scope. The TCP transport has no TLS layer, so it does not come cheaply from this code.

Tests

lib/velo/src/transports/quic/auth_tests.rs. Each refusal test has a positive control with the same client. "Nothing dispatched" is read off the listener's message, response, event and shutdown streams, which every routed frame reaches.

  • Refused before any frame: a raw client with no certificate; a raw client and a velo transport with a certificate from another CA; a raw client with a chain from the CA but another key; a default transport.
  • A raw server with a chain from the CA but another key is refused by the dialer, and gets no frame.
  • The listener's authorizer refuses a valid certificate. It was called with the peer's URI.
  • The dialer's authorizer refuses a listener: three queued frames fail, the listener gets nothing, the call names the instance id and lane 0, and the health check fails.
  • Positive control: A sends a Message, and B's first frame on the connection it dials back is the Response. Each authorizer saw one dialed and one accepted connection.
  • Positive control: a stream through the messenger mux on lane 3 (streaming/messenger_mux/lane_tests.rs). The lane 3 connections begin with mux traffic, and both nodes authorized their lane 3 dial.
  • Rotation, with the peer registered before each change: after rotate, a new lane presents the new certificate. After set_roots drops the old CA, a new lane with an old-CA certificate is refused while the open lane keeps working.
  • Velo level: B accepts A and refuses C, both from B's CA. A's unary call gets its answer, C's times out, and B's handler counted one call.
  • Mutations: with the listener gate, the dialer gate, or the TLS 1.3 signature checks removed, at least two tests fail each.

Conflicts

git merge-tree against #153 (advertised addresses) shows conflicts in README.md (both add a paragraph above the lanes paragraph) and in lib/velo/src/transports/quic/transport.rs (register and the struct fields). builder.rs, Cargo.toml and the book merge cleanly. Whichever PR merges second needs a small rebase. The hub TLS PR (#155) merges cleanly with both.

Review ledger

An independent reviewer did three rounds (thermo-nuclear-code-quality-review and rust-code-review). Each valid finding got a failing test or reproduction first.

Round Finding Ruling Change
1 High. Two set_roots calls at once can leave the dialer trusting one CA and the listener another, because the roots and the client verifier sat behind separate locks. Valid. concurrent_root_updates_leave_both_directions_agreeing failed at round 1378 of 2000. The roots and the client verifier are one Trust snapshot behind one lock.
1 Medium. rustls-webpki = "0.103" admits 0.103.7, which has no valid_uri_names. Valid for the requirement. The requirement is now 0.103.8, and the resolver refuses 0.103.7. No minimal-versions CI job: the repo has none, so that is a separate decision.
2 Medium. rustls = "0.23" admits 0.23.22, which has no CertifiedKey::from_der. Valid. A downstream fixture with only quic and rustls 0.23.22 failed with E0599. The requirement is now 0.23.23. The fixture refuses 0.23.22 and builds on 0.23.23.
3 None Verdict: ready

During development, a test found that TLS 1.3 session resumption presented the identity of the first session after rotate. Resumption is off in CA mode.

The QUIC transport pins a self-signed certificate per transport, and
the listener does not authenticate the dialer. This change adds a mode
in which both peers present certificates from a private CA (ACM
Private CA, SPIRE, cert-manager) and verify each other in the TLS
handshake. The listener requires a client certificate.

An application authorizer gets the verified peer identity (the leaf,
the chain, and its URI SANs, such as spiffe://deployment/class/cell)
and the connection origin. It decides on every connection, on every
lane, in both directions, after the handshake and before the transport
reads or writes a frame. A refused connection is closed with the code
REFUSED and dispatches nothing. The messenger's _hello cannot do this
check: system handlers skip it, and a connection that a peer dials back
can begin with a response.

QuicTls::rotate and QuicTls::set_roots change the certificate and the
roots without a restart. New handshakes use them, also to peers that
were registered before. Session resumption and 0-RTT are off in this
mode: a resumed session skips certificate verification and keeps the
first identity, which a test caught.

The pinned self-signed certificate stays the default.

Tests: a dialer without a certificate, with a certificate from another
CA, with a chain but not its key, and a default transport are refused
before any frame; a server chain without its key is refused by the
dialer; each authorizer refuses a valid certificate; a connection that
begins with a response and one that begins with mux traffic on lane 3
are authorized and delivered; rotation of the certificate and the
roots; a Velo handler counts no calls from a refused peer.

Signed-off-by: Ryan Olson <rolson@nvidia.com>
@coderabbitai

coderabbitai Bot commented Oct 22, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.98433% with 69 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
lib/velo/src/transports/quic/auth.rs 81.51% 47 Missing and 9 partials ⚠️
lib/velo/src/transports/quic/transport.rs 81.39% 7 Missing and 1 partial ⚠️
lib/velo/src/transports/quic/transport/builder.rs 84.37% 0 Missing and 5 partials ⚠️

📢 Thoughts on this report? Let us know!

set_roots wrote the roots (dialer) and the client verifier (listener)
under two locks. Two calls at once could interleave so that, after
both returned, the dialer trusted one set of roots and the listener
the other. Withdrawing a CA could then stay without effect in one
direction. Both now live in one snapshot behind one lock.

Also require rustls-webpki 0.103.8, the first release with
valid_uri_names. 0.103.7 met the old requirement and does not build.

Signed-off-by: Ryan Olson <rolson@nvidia.com>
…y::from_der

A consumer that kept rustls 0.23.22 met the old requirement and failed
to build velo with the quic feature.

Signed-off-by: Ryan Olson <rolson@nvidia.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant