Repository navigation
Apply hostname address mappings to proxy CONNECT targets - #2647
Conversation
Through a proxy it is the proxy, not the client, that opens the connection to the origin. SSLClient and WebSocketClient built the CONNECT target from host_ alone, so an address set with set_hostname_addr_map() took effect on direct connections only and was ignored through a proxy. When host_ is a name the proxy cannot resolve (e.g. an SNI-only name), the tunnel fails; when it resolves elsewhere, the tunnel reaches a different origin than a direct connection would. Add detail::make_connect_request_target(), which builds the CONNECT target from apply_addr_map(), and use it for SSLClient's CONNECT, its Digest retry, and WebSocketClient's CONNECT. The tunnelled request keeps host_ for its Host header and SNI, matching a direct connection.
- make_connect_request_target(): IPv4, IPv6 and hostname mappings, and the empty and absent mapping cases. - SSLClient: the CONNECT target is the mapped address, while SNI and Host inside the tunnel stay the hostname. - SSLClient: the CONNECT retried after a Digest 407 uses the mapped address too. - WebSocketClient: the CONNECT target and its Host are the mapped address.
|
Thanks for the PR. Before going further, I'd like to understand the need. 1. Do you need this now? 2. Existing behavior can't change 3. If we add it, this is the shape I have in mind
This is similar to how curl separates If we go this way, one more thing to cover: in Let me know about the use case first, and we can go from there. |
|
Thanks for the review! 1. Yes, I need it. I have to set the SNI separately from the server address, and now need the same connection to go through a proxy. In curl terms: The first one maps to 2. You're right, my mistake. 3. Your proposal sounds good. Following the curl
The calls line up one-to-one with curl. My case (always with a custom SNI). Without a proxy: httplib::SSLClient cli("sni.example", 443);
cli.set_connect_to_map({{"sni.example", "server.example"}});
// connects to server.example, SNI sni.exampleThrough a proxy, only the proxy setting is added: httplib::SSLClient cli("sni.example", 443);
cli.set_connect_to_map({{"sni.example", "server.example"}});
cli.set_proxy("proxy.example", 3128);
// CONNECT server.example:443, SNI sni.exampleThe users you mentioned (the proxy resolves the original name). Nothing changes for them: httplib::SSLClient cli("origin.example", 443);
cli.set_proxy("proxy.example", 3128);
cli.set_resolve_map({{"origin.example", "10.0.0.5"}}); // = set_hostname_addr_map()
// CONNECT origin.example:443 (the mapping is not used)Different addresses per path (10.0.0.5 for direct, y.example through a proxy). As in curl, httplib::SSLClient cli("origin.example", 443);
cli.set_connect_to_map({{"origin.example", "y.example"}});
cli.set_resolve_map({{"y.example", "10.0.0.5"}}); // = set_hostname_addr_map()
// direct: connects to 10.0.0.5 / with set_proxy(): CONNECT y.example:443Let me know if I've misread anything. Going with your |
set_hostname_addr_map() works like curl's --resolve: it applies to direct connections only, and through a proxy the proxy resolves the original hostname. Applying it to the CONNECT target, as the previous commits did, changes what existing users send through a proxy, so keep it as it was. Add set_connect_to_map(), which works like curl's --connect-to: it retargets the direct connection and the CONNECT through a proxy, while the SNI, the tunnelled Host header and certificate verification keep the original hostname. As in curl, it applies before the addr map on a direct connection, so the addr map can still resolve the rewritten name. Add set_resolve_map() as an alias of set_hostname_addr_map() so the pair reads like the curl options. Both setters are on ClientImpl, Client and WebSocketClient, and the connect-to map is copied to redirect clients like the addr map.
SSLClient builds the CONNECT Host header from host_, so with a connect-to entry the request read `CONNECT 127.0.0.1:443` with `Host: origin.example`. RFC 9112 3.2 expects the Host to match the request target, as curl and WebSocketClient already send it. When a connect-to entry retargets the CONNECT, set its Host to the same authority, for the first CONNECT and for the Digest retry. Without an entry the CONNECT request is unchanged.
- The CONNECT-target tests now use set_connect_to_map(), and also check that the CONNECT Host matches the target, for SSLClient (including the Digest retry) and WebSocketClient. - set_hostname_addr_map() alone keeps the original hostname as the CONNECT target and Host, for SSLClient and WebSocketClient. - On a direct connection, set_connect_to_map() changes the connection target and keeps the Host header, and set_resolve_map() applies to the name it rewrites to.
Replace the paragraph that described set_hostname_addr_map() as the CONNECT target with the curl --resolve / --connect-to split, and list the new setters in README-websocket.md.
|
Thanks for working through the options and for the curl comparison, that made the trade-off easy to see. After looking at both, I'd like to go with the explicit naming:
Each name says exactly when it applies, and the two maps are independent, so there is no ordering rule between them and the docs stay short. To get the same target on both paths, set both. A one-line note in the README that they correspond to curl's Since the old name doesn't say which path it applies to (which is how the Could you rework the PR along these lines? Most of what you have carries over:
Let me know if anything here doesn't work for you. |
Replace set_resolve_map() and set_connect_to_map() with two independent setters on ClientImpl, Client and WebSocketClient, as discussed in the review: - set_direct_hostname_addr_map(): the address the client connects to directly. This is what set_hostname_addr_map() does today. - set_tunnel_hostname_addr_map(): the address the proxy is asked to connect to in the CONNECT request. SNI, the tunnelled Host header and certificate verification keep the original hostname. The two maps no longer interact, so the connect-to-then-resolve helper is gone: the direct map alone picks the direct connection target, and the tunnel map goes straight to the CONNECT target builder. The CONNECT Host still matches the request target, including the Digest retry, and the redirect client still copies both maps. set_hostname_addr_map() is kept as a thin wrapper around set_direct_hostname_addr_map() with no change in behavior, and is marked [[deprecated]]. Tests, README, README-websocket and docs-src switch to the new name; one test keeps calling the old name, with the deprecation warning suppressed, to pin that it still works. A new test pins that the tunnel map leaves a direct connection alone.
|
Thanks! I’ve reworked the PR as you suggested. One thing I’d like to raise: with a proxy set, “the address the client connects to directly” could be read as the proxy server. But set_direct_hostname_addr_map(), like set_hostname_addr_map() before it, is applied only to the request host, never to the proxy address. In curl, --resolve also applies to the proxy address given with --proxy. For example, curl --resolve proxy.example:3128:127.0.0.1 --proxy http://proxy.example:3128 https://example.com/ connects to the proxy at 127.0.0.1. So the name set_direct_hostname_addr_map() may suggest that it also covers the proxy address, when it doesn’t. A --resolve equivalent for the proxy address would need to be a separate function. What do you think? |
|
Good catch, thank you. "Direct" is meant in the proxy sense (as in a PAC file's DIRECT): a connection that does not go through a proxy. So neither map applies to the proxy address, and I'd rather not add a setter for it, since passing an IP literal to Could you add a sentence to the README saying so? |
|
Added the README note as you suggested. Thanks! |
Shorten the comments added for the tunnel map, and add a test that a cross-host redirect client keeps sending its CONNECT to the mapped target.
|
Thanks for your excellent contribution! |
Summary
Apply
set_hostname_addr_map()mappings to proxy CONNECT targets inSSLClientandWebSocketClient.Use the mapped address for the proxy's connection to the origin, while preserving the original hostname for the
Hostheader and TLS SNI inside the tunnel, matching direct-connection behavior.Problem
When connecting through a proxy, the proxy—not the client—opens the connection to the origin.
Previously,
SSLClientandWebSocketClientbuilt the CONNECT target fromhost_alone. As a result, addresses configured withset_hostname_addr_map()took effect on direct connections but were ignored when connecting through a proxy.This could cause:
Changes
detail::make_connect_request_target()to build the CONNECT target usingapply_addr_map().SSLClient.SSLClient.WebSocketClient.Hostheader and TLS SNI inside the tunnel.set_hostname_addr_map()behaves with CONNECT proxies.For example, with
origin.examplemapped to127.0.0.1on port 443:origin.example:443127.0.0.1:443origin.exampleorigin.exampleorigin.exampleorigin.exampleFor WebSocket connections, the CONNECT request's own
Hostheader uses the mapped CONNECT target. This is separate from theHostheader of the request sent inside the tunnel.Tests
Add regression coverage for:
SSLClient:Hostheader inside the tunnel retain the original hostname.WebSocketClient:Hostheader use the mapped address.