Skip to content

governance: host deny lists cannot refuse an unverifiable host reading #23734

Open

Description

@iamwhatever

The host-matcher list scopes added for the Crew Enterprise Edition partial controls (#23716 capabilities.telemetry otlp_hosts, #23717 capabilities.feature_videos_download hosts, #23719 capabilities.decisions endpoints) ask every host a client could connect to for a URL (governance_parts.matching.url_policy_hosts). That covers the standard library, urllib3 and yarl readings, IP literals in canonical form, and non-ASCII names in IDNA form. A reading with no such form is asked as "".

"" matches no list entry, so an allow list refuses it and a deny list permits it. This is the same posture network.egress already has with its unverifiable item. So an allow list is the hard bound, and a deny list refuses only the spellings it can name.

What remains open:

  • A deny-mode ruleset cannot refuse a URL whose host no parser can canonicalize. Closing that needs the archetype to know whether a ruleset governs the item at all, for example a "refuse the unverifiable item whenever the scope carries any ruleset" rule in ScopedRuleset.permits or the evaluator. That touches every host scope, network.egress included, so it needs its own design.
  • Unicode list entries are compared in IDNA form by _match_host. A parse-time warning for a host entry that has no IDNA form, beside the existing dead-entry warnings in archetypes.py, would tell an author when an entry can never match.

Found during review of #23716 / #23717 / #23719, where the host-reading class came back in four rounds (backslash authority, percent-encoding, IDNA, ideographic dot, numeric IPs, scoped IPv6). Those PRs fix the concrete spellings and record this residual instead of chasing further spellings.

Update: the round-6 heads leave _match_host unchanged from main. Entries in these lists are documented as ASCII (xn--) hosts with no root dot, and IPs in canonical form. A parse-time warning (or normalization) for a host entry that is non-ASCII, carries a root dot, or is a non-canonical IP literal belongs here too.

Update (round 12): entry checks at load cover exact entries fully (non-ASCII, root dot, brackets, zone, port, %/@//, non-canonical IP). Glob entries get a character check only. An IP glob in a non-canonical numeric spelling (010.0.0.*, 0x0a.0.0.*, 1677721*, ::ffff:*, *:ffff:*, 2001:db8:0:0:*) loads and matches no reading, so as a deny it refuses nothing. Several rule attempts each refused valid globs (*, 10*, 01-collector*, 2001:db8:10:0:*). A sound fix needs a glob-aware IP canonicalizer, or a rule that IP globs are written in a fixed CIDR-like form.

Activity

  1. added
    tier:T3Changes an existing user experience; needs a one-pager
    parkedKept open as a request; not on the current plan
    on Oct 24, 2026
  2. iamwhatever commented on Oct 25, 2026

    @iamwhatever
    ContributorAuthor

    Add from #23716 (bolichen97, archetypes.py:411): _unmatchable_glob and the exact-entry _PLAIN_HOST_RE check refuse +, !, % and similar on purpose, but the message says "no host is asked with" them. That is no longer true, because url_policy_hosts now asks every ASCII reading. Reword it to "has a character outside a DNS name; deny it with a broader glob such as *.example", and add a test that a + entry fails to load.

  3. iamwhatever commented on Oct 25, 2026

    @iamwhatever
    ContributorAuthor

    Update: the character-rule part of this issue (+, !, % in list entries) is done in #23716 ee3eec4. The numeric IP-glob spelling gap stays open here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    parkedKept open as a request; not on the current plantier:T3Changes an existing user experience; needs a one-pager

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions