Build the allowlist without connection tracking where there is none - #108
Merged
Merged
Conversation
gVisor provides no connection tracking — neither `-m conntrack` nor the older `-m state` — so the allowlist could not be programmed there and a run asking for both a stronger runtime and an allowlist refused. That made prod and gVisor mutually exclusive. It turns out the allowlist does not need connection state in principle, only in the way it was written. Measured on Rocky Linux 10.2 with gVisor: with OUTPUT at default-DROP and no conntrack anywhere, a single `-m owner --uid-owner` rule carried a complete HTTPS request — DNS, handshake, TLS, response. Kernel- generated packets match owner there, which was the objection most likely to sink this. So the conntrack rules are now conditional, detected by trying them on a scratch chain rather than inferred from the runtime name. On an ordinary host nothing changes. Where they are unavailable, outbound filtering is unchanged: the proxy's uid is what admits its traffic, the guest's 80/443 is still redirected into it, and the redirected packets are loopback-bound and already accepted. What is lost is INBOUND filtering, because a stateless chain cannot distinguish a reply from an unsolicited connection — denying both breaks every allowed request, and admitting both is what a default-ACCEPT INPUT does anyway. The run says so plainly rather than leaving it to be discovered. The containment argument for that, which is why this is a limitation rather than a hole: nothing inside the container can answer an inbound connection, because the answer would leave from a uid the OUTPUT chain denies, so the handshake never completes. That argument is reasoning, not yet a measurement, and docs/roadmap/task-3-gvisor-egress.md says which parts are which. The IPv6 accept-replies rules go the same way and lose nothing: IPv6 is rejected in both directions regardless, so nothing could ever have been established.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The first version probed conntrack and shipped. The next run on the gVisor host
got past it and hit the wall behind it:
Warning: Extension limit revision 0 not supported, missing kernel module?
Warning: Extension LOG revision 0 not supported, missing kernel module?
iptables: Invalid argument.
sandbox-cli: egress firewall setup failed; refusing to run without the requested allowlist
which is the mistake this project already names: characterise the surface once
rather than fix one error at a time. The earlier probe answered conntrack,
owner, nat and REDIRECT, and never asked about `-m limit`, `-j LOG` or REJECT.
So the probe is now general — rule_ok tries any rule on a scratch chain — and
three things are asked before the first real rule is written:
- conntrack, as before;
- `-m limit` + `-j LOG`, which is the rate-limited record of what was refused.
Diagnostics rather than enforcement: without it the rules are identical and
only the explanation is missing, so it degrades. Worth knowing that denial
counts go with it;
- REJECT, which answers a blocked client with an ICMP error so it fails at once
instead of hanging to its timeout. Where it is missing, DROP denies exactly as
much — the difference is the courtesy, not the boundary.
Each degradation is announced when it applies, so a run on such a kernel says
what it is not doing rather than leaving it to be found. Nothing changes on an
ordinary host, where all three are present.
The feature probes ran against $IPTABLES and their answers were then applied to the ip6tables chains. The two families are resolved independently — a host can serve one from legacy and the other from nft — so a capability measured on v4 is not evidence about v6. It is the required branch that makes this bite. Where the container has a global IPv6 address the rules carry no `|| true`, deliberately, because IPv6 left unfiltered while sandbox-cli reports an allowlist is the one failure this block exists to prevent. So a v4 REJECT probe standing in for v6 on a backend without the target aborts the whole setup under set -e, and the run refuses over a target nobody asked about. probe6 asks $IP6TABLES itself, after it is resolved, and sets DENY6/CONNTRACK6 from the answer. The no-global-address branch is gated the same way: everything there is `|| true` so an ungated conntrack rule would only fail quietly, but two branches writing one chain from different premises is how one of them ends up wrong later.
…ublish loses Two things the stateless path got wrong. `[ "$CONNTRACK6" = 1 ] && rule || true` was used inside the *required* IPv6 branch, where every other rule deliberately has no `|| true` because IPv6 left unfiltered under a claimed allowlist is the failure that block exists to prevent. The `|| true` is there to survive a false condition, but it swallows a failed rule just as happily. if/fi tolerates only the case meant to be tolerated. And the INPUT fallback opened the chain without flushing it, and without mentioning what changed. --publish is the casualty: the published service runs as the sandbox user, OUTPUT admits only the proxy's uid, and with no ESTABLISHED,RELATED rule there is nothing under which its replies may leave. The port is reachable and the answer never arrives. That is the containment argument working exactly as written — but the carve-out above exists precisely so a dev server does not silently stop answering when someone adds --allow, and a stateless kernel brings that back by another route. Warned when ports were actually published, since otherwise there is nothing to lose.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes the
conntrackrules conditional, so the egress allowlist can beprogrammed on a kernel that has no connection tracking — which is what makes
--runtime runscusable with an allowlist at all, and unblocks the conflict in#88.
Why this is possible
The allowlist does not need connection state in principle, only in the way it
was written. Measured on the gVisor host,
OUTPUTat default-DROP, no conntrackanywhere:
A complete request — DNS, handshake, TLS, response — carried by the uid rule
alone. Kernel-generated packets match
ownerunder gVisor, which was theobjection most likely to sink this.
What changes, and what does not
Detected by trying the rule on a scratch chain, not inferred from a runtime
name, so nothing changes on an ordinary host.
Where conntrack is unavailable:
out; the guest's 80/443 is still REDIRECTed into it; the redirected packets
are loopback-bound and already accepted by the rule above.
from an unsolicited connection. Denying both breaks every allowed request;
admitting both is what a default-ACCEPT INPUT already does. The run prints
this rather than leaving it to be found.
in both directions regardless, so nothing could ever have been established.
The part that is still an argument
Why losing inbound filtering is a limitation and not a hole: nothing inside the
container can answer an unsolicited connection, because the answer would leave
from a uid
OUTPUTdenies, so the handshake never completes. That isreasoning, not a measurement.
docs/roadmap/task-3-gvisor-egress.md(#107)says which parts are which, and settling it needs two containers on one network.
Testing
Not run end to end by me — no daemon here. On the gVisor host:
Expect the backend line, the no-conntrack notice, and
200. Then the samewithout
--runtime runscto confirm an ordinary runc run is untouched — nonotice, and the INPUT chain still default-deny.