Repository navigation
governance: host deny lists cannot refuse an unverifiable host reading #23734
Description
The host-matcher list scopes added for the Crew Enterprise Edition partial controls (#23716 capabilities.telemetry otlp_hosts, #23717 capabilities.feature_videos_download hosts, #23719 capabilities.decisions endpoints) ask every host a client could connect to for a URL (governance_parts.matching.url_policy_hosts). That covers the standard library, urllib3 and yarl readings, IP literals in canonical form, and non-ASCII names in IDNA form. A reading with no such form is asked as "".
"" matches no list entry, so an allow list refuses it and a deny list permits it. This is the same posture network.egress already has with its unverifiable item. So an allow list is the hard bound, and a deny list refuses only the spellings it can name.
What remains open:
- A deny-mode ruleset cannot refuse a URL whose host no parser can canonicalize. Closing that needs the archetype to know whether a ruleset governs the item at all, for example a "refuse the unverifiable item whenever the scope carries any ruleset" rule in
ScopedRuleset.permitsor the evaluator. That touches everyhostscope,network.egressincluded, so it needs its own design. - Unicode list entries are compared in IDNA form by
_match_host. A parse-time warning for ahostentry that has no IDNA form, beside the existing dead-entry warnings inarchetypes.py, would tell an author when an entry can never match.
Found during review of #23716 / #23717 / #23719, where the host-reading class came back in four rounds (backslash authority, percent-encoding, IDNA, ideographic dot, numeric IPs, scoped IPv6). Those PRs fix the concrete spellings and record this residual instead of chasing further spellings.
Update: the round-6 heads leave _match_host unchanged from main. Entries in these lists are documented as ASCII (xn--) hosts with no root dot, and IPs in canonical form. A parse-time warning (or normalization) for a host entry that is non-ASCII, carries a root dot, or is a non-canonical IP literal belongs here too.
Update (round 12): entry checks at load cover exact entries fully (non-ASCII, root dot, brackets, zone, port, %/@//, non-canonical IP). Glob entries get a character check only. An IP glob in a non-canonical numeric spelling (010.0.0.*, 0x0a.0.0.*, 1677721*, ::ffff:*, *:ffff:*, 2001:db8:0:0:*) loads and matches no reading, so as a deny it refuses nothing. Several rule attempts each refused valid globs (*, 10*, 01-collector*, 2001:db8:10:0:*). A sound fix needs a glob-aware IP canonicalizer, or a rule that IP globs are written in a fixed CIDR-like form.
Activity
fix(governance): ask an odd ASCII host as read, so a deny glob still …
fix(governance): ask an odd ASCII host as read, so a deny glob still …
fix(governance): ask an odd ASCII host as read, so a deny glob still …
fix(governance): ask an odd ASCII host as read, so a deny glob still …
fix(governance): ask an odd ASCII host as read, so a deny glob still …
fix(governance): ask an odd ASCII host as read, so a deny glob still …
iamwhatever commentedon Oct 25, 2026
Add from #23716 (bolichen97, archetypes.py:411):
_unmatchable_globand the exact-entry_PLAIN_HOST_REcheck refuse+,!,%and similar on purpose, but the message says "no host is asked with" them. That is no longer true, becauseurl_policy_hostsnow asks every ASCII reading. Reword it to "has a character outside a DNS name; deny it with a broader glob such as *.example", and add a test that a+entry fails to load.iamwhatever commentedon Oct 25, 2026
Update: the character-rule part of this issue (
+,!,%in list entries) is done in #23716 ee3eec4. The numeric IP-glob spelling gap stays open here.