Repository navigation
Conversation
The hub's public and control listeners served plain HTTP only. Each can now serve TLS, configured by the program that builds the hub: HubBuilder::public_tls and HubBuilder::control_tls take a HubTls. The library reads no files; the program passes DER certificates and keys. Plain HTTP stays the default. HubTls::client_certs asks clients for a certificate and verifies it against the given roots, Required or Optional. The verified ClientIdentity (leaf, chain, URI SANs) is available to routes through an extractor (401 without one, or Option<ClientIdentity>), and to registration hooks as RegistrationView::client, Txn::client and Committed::client, so an extension can admit only the identities that it authorizes. HubTls::rotate and HubTls::set_client_roots apply to new connections without a restart. TLS 1.3 only, with no session resumption, because a resumed session skips the client certificate check and keeps the first identity. Each handshake runs in its own task under a 10 s timeout, so a silent client does not hold up accept. Both listeners serve with connect info, so routes read the identity the same way on plain and TLS listeners. Tests: the existing HubClient over TLS through http_client; a client without a certificate, or with one from another CA, is refused when verification is on; an extension route and every registration hook see the identity, and admit refuses an identity it does not authorize; the service watch (SSE) over TLS; rotation of the certificate and the client roots with one client config kept throughout; a silent client does not block other handshakes. Signed-off-by: Ryan Olson <rolson@nvidia.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Comment |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Three faults in the TLS listener: - It accepted from the moment it was built, before the hub served, and each finished handshake then waited in a task of its own with no limit. The listener now accepts only inside axum's accept, and at most MAX_PENDING_TLS (256) connections are in a handshake or waiting for axum. Past that it stops accepting, and the kernel backlog queues the rest, as for a plain listener. - A detached task owned the listening socket, so shutdown could return while the port was still bound, up to a second after an accept error. The listener now owns the socket, and it closes when axum::serve drops the listener, before the server task that the hub joins ends. - ALPN offered h2, which the hub does not build axum support for. It offers http/1.1 only. Also require rustls 0.23.23, the first release with CertifiedKey::from_der. Signed-off-by: Ryan Olson <rolson@nvidia.com>
Signed-off-by: Ryan Olson <rolson@nvidia.com>
Signed-off-by: Ryan Olson <rolson@nvidia.com>
Signed-off-by: Ryan Olson <rolson@nvidia.com>
This was referenced Oct 22, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The hub's public and control listeners can serve TLS, with optional client certificate verification. The verified client identity is available to routes, extensions, and the registration path. Plain HTTP stays the default.
HubTls::new(chain, key), andHubBuilder::public_tls/HubBuilder::control_tls. Each listener has its own setting, so a common setup is server TLS on the public listener and client certificates on the control listener. The library reads no files. The program passes DER certificates and keys (velo_hub::pki_types, re-exported from rustls).HubTls::client_certs(roots, ClientCerts::Required | ClientCerts::Optional). Required: a client without a certificate that verifies fails the handshake. Optional: an anonymous client connects, but a certificate that it presents must verify.ClientIdentity(leaf, chain, URI SANs):unauthorized(401) without a verified certificate.Option<ClientIdentity>serves anonymous clients too.RegistrationView::client,Txn::client,Committed::client. An extension that admits only some identities refuses inadmit.HubTls::rotateandHubTls::set_client_roots. A resolver and a verifier read the current values on each handshake, so new connections use them without a restart.accept. So it takes no connection before the hub serves, and its port is free whenshutdownreturns.HANDSHAKE_TIMEOUT(10 s), so a client that connects and sends nothing does not hold up the next one.MAX_PENDING_TLS(256) connections are in a handshake or wait for axum. Past that, the listener stops accepting, and the kernel backlog holds the rest, as for a plain listener.into_make_service_with_connect_info), so routes read the identity the same way on plain and TLS listeners.The client side needs no change. A program builds a
reqwest::Clientwith TLS, and with a client certificate if needed, and passes it toHubClientBuilder::http_clientwith.scheme("https").Dependencies
Under
server:rustls>= 0.23.23 andtokio-rustls(ring provider, no process-wide default provider), andrustls-webpki>= 0.103.8 for URI SANs. All three are already in the lockfile. Dev: reqwest getsrustls-tls-manual-roots(ring), and rcgen makes the test certificates. With the no-provider reqwest feature, every defaultreqwest::Clientin the other suites panicked, so the dev feature includes a provider.velo-hub and velo-hub-ext were never published, so the semver gate skips them.
cargo tree -p velo-hub-extshows noveloand noprometheus.RegistrationView,Txn, andCommittedget a new public field, which is a break for code that builds them. Only the hub builds them.Tests
lib/velo-hub/tests/tls.rs:HubClientover TLS on both listeners throughhttp_client: register, discover, unregister. A plain-HTTP request to a TLS listener gets no answer.ClientIdentity. Without a certificate, it answers 401, and theOptionroute answers "anonymous".admitrefuses an identity that the extension does not authorize, andadmit,prepareandcommiteach saw the identity of the admitted client.Added.rotate(the new certificate is served) andset_client_roots(the old-CA client is then refused, and a new-CA client is accepted).startholds the hub's startup.h2andhttp/1.1getshttp/1.1.shutdown.newandrotate, andset_client_rootswithoutclient_certs, are refused.All velo-hub suites pass with
--all-features, except the etcd scale testa_prefix_at_target_scale_loads. On this machine, that test fails onorigin/maintoo, in 3 of 3 runs. It is not from this PR.Review ledger
An independent reviewer did four rounds (
thermo-nuclear-code-quality-reviewandrust-code-review). Each valid code finding got a test that failed first.no_tls_handshake_completes_before_the_hub_servesandpending_tls_connections_are_bounded.accept. A semaphore of 256 slots covers the connections in a handshake or waiting for axum.h2, which a hub built with onlyservercannot serve.the_listener_negotiates_http_1_1.http/1.1only.shutdowncould return while a detached accept task still held the TLS port, for up to 1 s after an accept error.dropping_a_tls_listener_closes_its_socket_at_once.HANDSHAKE_TIMEOUT" is not true: the timer starts at accept, and the backlog has no deadline.From the V1 review: the
rustlsrequirement went up to 0.23.23 (CertifiedKey::from_der).