feat(fftls,ffresty,ffdns,ffnet) Networking Metrics and IP/Server Configs#218
feat(fftls,ffresty,ffdns,ffnet) Networking Metrics and IP/Server Configs#218onelapahead wants to merge 6 commits into
Conversation
Signed-off-by: hfuss <hayden.fuss@kaleido.io>
…work as expected - CA bundles make metrics for all certs, leaf only the leaf w/ a key makes a metric Signed-off-by: hfuss <hayden.fuss@kaleido.io>
|
Approved |
Signed-off-by: hfuss <hayden.fuss@kaleido.io>
onelapahead
changed the title
Signed-off-by: hfuss <hayden.fuss@kaleido.io>
EnriqueL8
left a comment
EnriqueL8
left a comment
There was a problem hiding this comment.
Thanks @onelapahead ! A few comments
| // HTTPMaxIdleConnsPerHost the max number of idle connections per host | ||
| HTTPMaxIdleConnsPerHost = "maxIdleConnsPerHost" | ||
|
|
||
| // HTTPDNSServers an optional list of DNS server addresses (host or host:port, port defaults to 53) to use for |
There was a problem hiding this comment.
Are there any default DNS server addresses built into GO Dns resolver already?
There was a problem hiding this comment.
sorta - they just use the /etc/resolv.conf of the underlying OS (assuming Unix) and the equivalent for Windows by default.
So netgo is its own DNS resolver, but in terms of what nameservers to use it still gets that from the host, so that it operates w/ the proper control prescribed by the network admin.
Thats how Go microservices in K8s (using netgo 98% of the time in my personal experience) just magically know how to get the cluster CoreDNS IP address(es) for DNS resolution w/o any additional config.
| // NewDNSResolver builds a pure-Go *net.Resolver that dials the given DNS servers | ||
| // (each host or host:port, port defaulting to 53) in order, failing over to the | ||
| // next on error. Returns nil when no servers are given (use the system resolver). | ||
| // Exported so non-ffresty dialers — e.g. a WebSocket dialer — can honour the same |
There was a problem hiding this comment.
Yeah this exported reason makes it a different file or even pkg
There was a problem hiding this comment.
eek I see - maybe ffdns
onelapahead
changed the title
Signed-off-by: hfuss <hayden.fuss@kaleido.io>
onelapahead
changed the title
| // DefaultDeniedCIDRs are blocked by default to mitigate SSRF: loopback, link-local (including | ||
| // the cloud metadata endpoint 169.254.169.254 and the AWS IMDS IPv6 endpoint), unspecified / | ||
| // "this host", and multicast / reserved / broadcast ranges. Private RFC1918 / IPv6 ULA ranges | ||
| // are intentionally NOT included — these dialers are commonly used for legitimate internal | ||
| // service-to-service calls, so blocking private space is deferred to network firewalls / | ||
| // zero-trust rather than baked in (callers wanting that can use additionalDeniedCIDRs). | ||
| var DefaultDeniedCIDRs = []string{ | ||
| "0.0.0.0/8", // unspecified / "this host" (RFC 1122) | ||
| "127.0.0.0/8", // IPv4 loopback | ||
| "169.254.0.0/16", // IPv4 link-local, incl. cloud metadata 169.254.169.254 | ||
| "224.0.0.0/4", // IPv4 multicast | ||
| "240.0.0.0/4", // IPv4 reserved (incl. 255.255.255.255 broadcast) | ||
| "::1/128", // IPv6 loopback | ||
| "::/128", // IPv6 unspecified | ||
| "fe80::/10", // IPv6 link-local | ||
| "fd00:ec2::254/128", // AWS IMDS IPv6 endpoint (cloud metadata) | ||
| "ff00::/8", // IPv6 multicast | ||
| } |
There was a problem hiding this comment.
Note - by default we do not include private IPs, bc for most use cases ffnet/ffresty is being used for private IP connectivity.
The burden is on the user to restrict it further to private IPs or whatever expected ranges, or relax it if loopback hosts are expected as a different example.
Disclaimer - we always recommend zero trust (mTLS) and/or high-trust (firewall) networking beneath these clients as the primary line of defense.
There was a problem hiding this comment.
I think where we provide a pre-baked config that can be enabled, it should be an enum to switch off all private IP ranges, including link-local.
I don't think it should be a default for all uses of ffresty.
There was a problem hiding this comment.
Agreed - so we do not default to denying any, but there are now a bunch of IANA lists of IPs that ffnet users can import to build their own denylist's easily.
onelapahead
changed the title
Signed-off-by: hfuss <hayden.fuss@kaleido.io>
4 participants
fftls
Like
ffrestyand other pacakges - a static registration of a metrics registry makes a shared metrics manager subsystem and a single metric withissuer,subject, andtypelabels. Where thefloat64value of the gauge is the Unix timestamp of the expiry date (common pattern in Prom metrics like in FF DX metrics and Besu/Erigon timeSinceLastBlock).ffdns
A new config and resolver builder for customizing DNS timeouts and (name)servers for lookups. With metrics if enabled (though I don't think most users want DNS metrics always on as its expensive).
ffnet
A custom
Dialerwith a configurable CIDR denylist for IP ranges to reject. Defaults to empty, but provides exported vars with all the IANA IP ranges that users may want to deny in certain client configs.ffresty
Config option for
dnsServersto allow for customizing DNS resolution to use overridden DNS servers. Useful for split horizon DNS or other enterprise networks (though deferring to the network settings in Kubernetes/cloud/on-prem environments is still preferred). Additionally, for programmatic users, the ability to provide your own custom resolver.Note that previously we never prescribed netgo - letting the compile time of the runtime determine what DNS implementation is used. When DNS servers is used, netgo must be used and is therefore specified.