Skip to content

Give tests a dead endpoint that keeps its port, and move the sixteen closed-server sites onto it - #1472

Merged
jmrplens merged 14 commits into
issue-1106-collector-examplefrom
issue-1132-dead-endpoint
Oct 21, 2026
Merged

jmrplens merged 14 commits into
issue-1106-collector-examplefrom
issue-1132-dead-endpoint

Conversation

@jmrplens

@jmrplens jmrplens commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

This is the first layer of #1132: the helper and the sites. The gate that keeps the pattern out comes in the next layer, and the Makefile, CI and documentation wiring in the one after it.

A test that needed an instance that is gone started an httptest server and closed it, keeping the URL. Closing it gives the port back to the kernel at once, and whatever is given that port next can answer the request the test exists to see fail: a parallel test of the same package starting a server of its own, or a test binary of another package running beside it in the same coverage run. #1131 fixed one such flake in cmd/bench_resources; this moves every other site.

internal/testutil/netfixture has two things. DeadEndpoint(t) holds a loopback port for the whole test and closes every connection it accepts before reading a byte, and its cleanup waits for the accept loop to end. An accept that fails for any reason but the listener being closed, such as a process out of file descriptors while several test binaries run at once, is retried after a pause, as net/http's server does: a loop that ended there would keep the port with nothing accepting, and the kernel would queue every connection made to it, so a request would wait out its deadline instead of failing at once. NewStoppableServer(t, handler) answers through its handler until Stop, then hangs up on every request, the ones on a connection a client kept from before included, closes the connections already open, and keeps its port. Each gives its port back in the cleanup it registers, which runs after every cleanup registered later and before any registered earlier, so a cleanup registered before either is asked for must not use its address; their doc comments say so. The bench package's closedEndpoint moves onto DeadEndpoint.

The issue asked for the helper in internal/testutil, but the code does not allow it: internal/testutil imports internal/gitlab, so internal/gitlab's own tests, which hold four of the sites, cannot import it (import cycle not allowed in test). The package sits beside testutil instead, the way internal/testutil/hints does for the same reason, imports nothing of this module, and is added to the test support TestDependencies_TestSupport_NeverReachesTheServerBinary keeps out of the server binary.

The sites are the sixteen the gate in the next layer finds on this branch's parent, in seven packages. Fourteen are the issue's own rows that still exist, at the line numbers they have moved to: the issue lists fifteen, and TestCredentialRejected_OnlyAnExplicitRefusalCountsAsOne went away with #1139, so the issue's three credential tables in internal/gitlab are two now (TestCheckCredential_ThreeAnswers_KeepsNoVerdictApart is TestCheckCredential_FiveAnswers_KeepsEachApart). The other two arrived with #1146, after the issue was opened: TestAdmitToken_AssumedScopes_SaySoInTheAdmission in internal/oauth and TestDetectToken_NoAnswer_IsNothingKnown in internal/gitlab/scopes_test.go. One more of the issue's rows changed shape before this: TestRunProbe_ExplicitTarget's "nothing listening" case had been moved into a sequential test of its own (#1258) because the reset it could meet broke its refused assertion. I read each site before changing it.

As decided on the issue on 2026-10-17, the probe stops asserting refused: an accepted and closed connection ends with an EOF or a reset. The case goes back into the parallel table on a dead endpoint and holds what the probe is for, an unhealthy exit that names the target that gave no answer, and the sequential test is gone. The comments that described a refused connection in internal/oauth, internal/gitlab and cmd/internal/apidocs now say what happens.

TestNewGitLabVerifier_NetworkErrorWithCache is the one site that needs an instance to answer first and then stop, with the verifier and the cache bound to its URL, so it uses NewStoppableServer. Its comment claimed a failed re-validation removes the cached entry, but the test deletes the entry itself, standing in for its expiry, since a cached identity is answered without asking the instance. The comment now says what the test holds: a re-verification against an instance that stopped answering admits nothing and caches nothing. The two credential tables in internal/gitlab built their client on a live server and closed it for the unanswered row; building a client sends nothing, so that row now builds it on the dead address from the start, through a small credentialEndpoint helper, and internal/serverpool does the same for its unreachable row.

Three packages whose tests this layer touches were below the bar on main, and five commits close them. cmd/audit_graphql_documents (97.0% of statements) ran main only in a child process, whose coverage counters are not the test's: main now hands its arguments, the environment and both streams to runMain and ends the process through exitProcess, and two tests in the test process cover both; the child-process tests stay. test/e2e/modeleval/internal/provider (96.4%) had failure paths held for one adapter and not the others, a few cases no test built, and four branches no run can reach: those are reached through the catalog builder handed to buildFakeCatalog, the encodeCall variable and the cached catalog error, or removed where the code could not fail (a second decode in producedValue, an encoding error in normalizeSchema), and its unknown-provider test now holds the refusal it is named for. At 100% of statements it still had twenty conditions evaluated one way only and 28 mutants lived and 21 uncovered, all in code this layer did not otherwise change; a commit of its own answers each of them, with tests for what nothing held (the echo checks of all three adapters, a call named for its position, the wrong-args variant planting one fact, an individual tool name a sibling took, every side of the HTTP status classification and of the price window, the latency in milliseconds, the default client's timeout), four switches with no tag rewritten as ifs because gremlins cannot see into their case expressions, and two checks removed that could never decide anything. cmd/internal/apidocs had a loop condition that was never false, since the loop always left through the break after its last attempt, a status classification in that loop written as a switch with no tag, whose six case mutants gremlins reported uncovered although each dies when applied by hand, and a const group whose one exported name go doc read as undocumented, so the godoc audit reported it. One commit fixes the loop and rewrites the classification as a chain of ifs gremlins can measure, and another the const group. The testing README's account of those six mutants now says so.

The godoc audit also reported, in packages this layer touches, three comments go doc does not read where they stand, none of them new here: a second package comment in cmd/bench_resources, because fairness.go's file comment sat directly on its package clause; StartIdleEviction's comment in internal/serverpool, run into recoverSweep's two functions above it, which left the method undocumented; and, under --include-tests, eleven tests of cmd/internal/apidocs, the one this layer moves among them, and three of internal/serverpool with no doc comment at all. The last commit puts each comment on what it documents.

How I tested it. The netfixture tests were written first and failed to build on the parent (undefined: DeadEndpoint, undefined: NewStoppableServer), and the accept-loop test failed on the loop that ended at the first failed accept (read pipe: i/o timeout, want the EOF of a hang-up). They pass now, 50 times in a row under -race. They hold that every request to a dead endpoint ends without an answer and promptly, that its port cannot be listened on while the test runs and is free once it ends, that the accept loop survives a failed accept and still ends once the listener is closed, that a stopped server hangs up on a kept connection and on a new one and is never reached again, that Stop cuts a response being streamed, and that a listener that cannot be opened fails the test naming why. The package is at 100% of statements, 10/10 conditions under gobco, and 4 mutants killed with none lived or uncovered. The new runMain tests failed to build on their parent (undefined: runMain, undefined: exitProcess); cmd/audit_graphql_documents is now at 100% of statements, 154/154 conditions, and 88 mutants killed with none lived or uncovered. test/e2e/modeleval/internal/provider is at 100% of statements, 406/406 conditions, and 235 mutants killed with none lived; the two it reports uncovered are the operators of the constant 180 * 24 * time.Hour that sizes the price window, which carries no statement counter, and the price tests hold that window to 180 days at its edge and past it. cmd/internal/apidocs is at 100% of statements and 110/110 conditions, and with the switch gone its mutants went from 62 killed, 4 lived and 10 uncovered to 67 killed, 4 lived and 4 uncovered, every survivor of a kind docs/development/testing/README.md records under "Reading a survivor": the four lived are the three boundaries whose sides agree and the clock boundary that page names for this package, and the four uncovered are operators in constant initializers. The packages this layer touches only in their tests are at the bar too. internal/oauth is at 224/224 conditions and 114 mutants killed with none lived or uncovered; internal/gitlab at 708/708 and 359 killed, none lived and 8 uncovered; internal/serverpool at 418/418 and 215 killed, none lived and 6 uncovered. The fourteen uncovered are operators in package-level constant and variable initializers, such as 30 * time.Second and config.EnvPrefix + "ALLOW_PRIVATE_INSTANCES", which carry no statement counter: the tool artifact the same page records. cmd/server and cmd/bench_resources are at 2270/2270 and 1544/1544 conditions. This layer changes only their tests (and a blank line in fairness.go), and a mutation run of either whole package means thousands of mutants each running the package's whole suite, so I measured the files those tests exercise: cmd/server's probe.go has 83 mutants killed, none lived and 2 uncovered, the operators of its two timeout constants, and cmd/bench_resources' sessions.go and held.go have 134 killed with none lived or uncovered. Every moved test passes under -race, every touched package passes in full under -race and is at 100% of statements, and golangci-lint with every e2e tag is clean on all of them. The gate in the next layer, run on this branch's parent, reports exactly these sixteen sites and exits 1; on this branch it exits 0.

Refs #1132

…p their port

A test that needs a server that is gone has been starting an httptest
server and closing it, keeping the URL. Closing it frees the port, and
whatever is given that port next can answer the request the test exists
to see fail.

DeadEndpoint holds a loopback port for the whole test and closes every
connection it accepts before reading a byte. An accept that fails for
any reason but the listener being closed, such as a process out of file
descriptors, is retried after a pause, since a loop that ended there
would leave the port held with nothing accepting and every request
waiting out its deadline. StoppableServer answers through its handler
until Stop, then hangs up on every request, the ones on connections kept
from before included, and keeps its port. Each gives its port back in
the cleanup it registers, which runs after every cleanup registered
later and before any registered earlier, as their doc comments say, so
a cleanup registered before either is asked for must not use its
address.

The package sits beside internal/testutil rather than inside it because
internal/gitlab's own tests need it and testutil imports internal/gitlab.
It is added to the test support the server binary must never link.

Refs #1132
…t is gone

closedEndpoint was the first dead endpoint that kept its port, written
for #1131. It moves onto the shared helper, which does the same thing
and waits for its accept loop to end before the test does.

Refs #1132
…ed port

TestRunProbe_ATargetNothingListensOn_IsReportedRefused and
TestRunProbe_Discovery took their "nothing answers" address from an
httptest server they closed. The first ran alone, so only another
process, such as a test binary of another package in the same coverage
run, could be given its freed port; the second runs in parallel, so a
parallel test of this package starting a server could be given it as
well. Either could then answer the probe.

The first was a test of its own, outside TestRunProbe_ExplicitTarget's
parallel table, because a sibling starting a process could make its
connect end in a reset that broke its "refused" assertion. The case
moves back into that table on a dead endpoint and stops asserting
"refused": a connection that is accepted and then closed ends with an
EOF or a reset. It now holds what the probe is for, an unhealthy exit
naming the target that gave no answer, and the separate test is gone.
Discovery's peer that answers nothing is a dead endpoint as well.

Refs #1132
Six verifier tests closed an httptest server to get an instance that
does not answer, all six of them in parallel, so another test of the
package could be given the port in between. Five now use a dead
endpoint, and their comments no longer describe a refused connection.

TestNewGitLabVerifier_NetworkErrorWithCache needs its instance to
answer once and then stop, and its verifier and cache are bound to the
URL, so it uses a stoppable server. Its comment now says what it checks:
the test deletes the cached entry itself, so what it holds is that a
failed re-verification admits nothing and caches nothing.

Refs #1132
…e start

TestInitialize_ServerDown and TestDetectToken_NoAnswer_IsNothingKnown
closed a server to get an unreachable URL, the first one from
stubVersionServer. The two credential probe tables built their client
on a live server and closed it for the unanswered row.

Building a client sends nothing, so those rows now build it on a dead
endpoint, through credentialEndpoint, and the other two take the dead
endpoint directly.

Refs #1132
…ble row

TestRevalidateAll_EvictsOnlyOnCredentialVerdict closed its server for
the unreachable instance, leaving a port another listener could answer
on with a status the row does not expect.

Refs #1132
…ests

Three sequential tests closed an httptest server to get an instance that
cannot be reached. They now take a dead endpoint, and the apidocs test
no longer calls the failure a refused connection.

Refs #1132
…unMain

main was only ever run in a child process the tests start, and the
coverage counters of a child are not the test's, so the package stood at
97.0% of statements with main at none. main now hands its arguments, the
environment and both streams to runMain, which parses a flag set of its
own and returns the status, and ends the process through exitProcess.

The child-process tests stay: they hold the streams and the status as
make and CI read them. Two tests in the test process cover runMain's
three ways out (a flag nobody defined, a request for help, a run) and
main's hand-off of the status. The usage line now names the command
rather than the path of the binary go run built.

Refs #1132
The provider package stood at 96.4% of statements. Most of what no test
reached was a failure path of one adapter that another adapter's tests
already held: a body that is not JSON was held for Anthropic only and is
now held for OpenAI and Google too, and the transport test runs over all
three. The rest is covered where it lives: an answer whose body ends
before its declared length, a request that cannot be built (a payload
that does not encode, an endpoint that does not parse), an empty option
in a spec, a result that is not JSON and a field an earlier step
produced read through resolveTruth, the text block of an assistant turn
that is not sent as a call, and the approval of a destructive action on
the individual surface, which the test's own comment promised and did
not check.

The unknown-provider case of New was refused for its missing key before
the provider was looked at, so it held nothing about the provider; it
is now given a key and holds the refusal's wording.

Four branches no input of a run can reach are now reachable or gone.
buildFakeCatalog is handed the catalog builder and the fake encodes its
calls through the variable encodeCall, so a test can make either fail,
and a test replaces the cached catalog error to reach the path that
reads it. producedValue decodes the result once and walks the decoded
value, where it decoded every level again and then decoded the value it
had just decoded a last time, and normalizeSchema returns what
json.Marshal answers, since encoding what was just decoded cannot fail.

Refs #1132
The loop in Fetcher.request counted its attempts in its own condition
and also broke out after the last one, so the condition was never false
and gobco reported it as evaluated one way only. The loop now has no
condition and ends through that break, which is the one way out it ever
took.

The status the loop classifies was a switch with no tag, whose case
expressions gremlins cannot see into, so six mutants of it were
reported uncovered although each dies when applied by hand. It is a
chain of ifs now, with retriedStatus naming the statuses worth asking
again for, and gremlins measures and kills each of them; the testing
README's account of those six says so.

Refs #1132
… its mutants

The provider package was at 100% of statements, but gobco found twenty
conditions evaluated one way only, and gremlins left 28 mutants lived
and 21 uncovered. Every one is now answered.

What the tests did not hold, they hold now: the echo of each adapter is
handed back only for an assistant turn and only when it is a non-empty
list of blocks, a content whose parts decode, or a message naming its
role; a rebuilt Anthropic call goes back as the call the model made;
a block or a part the record has no kind for is passed over; a Google
call the provider did not name is named for its position counted from
one, and so is the fake's call for its step; an answer with no
candidate names a block reason only when there is one; arguments of
null are a malformed call; the wrong-args variant plants the first fact
alone and the perfect one plants nothing; a meta call with no argument
carries no params; an individual tool name is kept only for the action
the call identifier resolves it to, tested through a catalog the test
builds, since the real one has no name a sibling took; a produced value
is refused for a step before the first; every side of the HTTP status
classification, a detail of exactly the bound's length, the default
client's timeout and the given client, the latency measured in
milliseconds, a free input or output alone, a price read this instant
or exactly at the window's edge, the age and the window in days, and a
temperature of two, below zero, or with two decimals.

Four switches with no tag are chains of ifs now (failureStatus, the
price date check, googleBlocks and resolveTruth), because gremlins
cannot see into the expressions of their cases and reported each of
them as uncovered whatever the tests held. Two checks that could never
decide anything are gone: a Google call's args are checked for length
after json.Valid, which already refuses empty input, and an echoed
content's parts were checked for length before a decoding that refuses
them empty. The note of the Gemini price is one literal, since an
operator in a package-level initializer carries no statement counter.

What remains is two uncovered mutants in the constant 180 * 24 *
time.Hour of maxPriceAge, for the same reason; the price tests hold the
window to 180 days at its edge and past it.

Refs #1132
go doc reads the comment of a const group, not of the specs in it, for
the one exported name a group declares, so the godoc audit reported
DefaultAreasURL as undocumented although its spec carries a long
comment. The group now says what its constants are for.

Refs #1132
The godoc audit reported three things in packages this layer changes,
none of them new here. cmd/bench_resources had a second package comment,
because fairness.go's file comment sat directly on its package clause;
a blank line makes it the file comment it is meant to be, and the
package comment stays in doc.go. internal/serverpool's
StartIdleEviction had its comment two functions above it, run into
recoverSweep's, so the method was undocumented and recoverSweep's
comment opened with another function's text; it is back on the method. Under
--include-tests, eleven tests of cmd/internal/apidocs, the one this
layer moves onto a dead endpoint among them, and three of
internal/serverpool had no doc comment; each now says what it holds.

Refs #1132
@sourcery-ai

sourcery-ai Bot commented Oct 21, 2026

Copy link
Copy Markdown

Sorry @jmrplens, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 5 days and 7 hours by commenting @sourcery-ai review. Upgrade to get a review now.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jmrplens has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 57 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: jmrplens/gitlab-mcp-server/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6f99575a-7a5a-4864-9f5f-6344b01a8d20
📥 Commits

Reviewing files that changed from the base of the PR and between 82e4271 and e5c9188.

📒 Files selected for processing: 33
  • cmd/audit_graphql_documents/main.go
  • cmd/audit_graphql_documents/main_test.go
  • cmd/bench_resources/fairness.go
  • cmd/bench_resources/held_test.go
  • cmd/bench_resources/sessions_test.go
  • cmd/internal/apidocs/api_docs.go
  • cmd/internal/apidocs/api_docs_test.go
  • cmd/internal/apidocs/areas.go
  • cmd/server/main_test.go
  • cmd/server/probe_test.go
  • docs/development/testing/README.md
  • internal/gitlab/client_test.go
  • internal/gitlab/scopes_test.go
  • internal/oauth/verifier_test.go
  • internal/serverpool/pool.go
  • internal/serverpool/pool_test.go
  • internal/serverpool/token_test.go
  • internal/testutil/netfixture/dead_endpoint.go
  • internal/testutil/netfixture/dead_endpoint_test.go
  • internal/testutil/netfixture/doc.go
  • internal/testutil/netfixture/stoppable_server.go
  • internal/testutil/netfixture/stoppable_server_test.go
  • test/e2e/modeleval/internal/provider/anthropic_test.go
  • test/e2e/modeleval/internal/provider/fake.go
  • test/e2e/modeleval/internal/provider/fake_test.go
  • test/e2e/modeleval/internal/provider/google.go
  • test/e2e/modeleval/internal/provider/google_test.go
  • test/e2e/modeleval/internal/provider/openai_test.go
  • test/e2e/modeleval/internal/provider/prices.go
  • test/e2e/modeleval/internal/provider/prices_test.go
  • test/e2e/modeleval/internal/provider/provider.go
  • test/e2e/modeleval/internal/provider/provider_test.go
  • test/e2e/modeleval/internal/provider/spec_test.go
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@jmrplens
jmrplens added this pull request to stack #1475 October 21, 2026 12:22
@jmrplens jmrplens added this to the 3.4.0 milestone Oct 21, 2026
@jmrplens jmrplens changed the title issue 1132 dead endpoint Give tests a dead endpoint that keeps its port, and move the sixteen closed-server sites onto it Oct 21, 2026
@github-actions github-actions Bot added security Security-related issue transport stdio and HTTP transports, the server process, and the transport e2e modules tooling The audit and generator commands under cmd/, and the Makefile targets that run them e2e The end-to-end suite and its Docker fixtures labels Oct 21, 2026
The base URL was a string concatenation with a literal scheme, which
SonarCloud reads as a request sent over plain HTTP. It is a loopback
address no request is meant to reach, so it is built as a url.URL, the
form the bench harness uses for its own loopback servers, and reads the
same. The local variable that held it is renamed so it no longer shadows
the url package.

Refs #1132

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jmrplens has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@sonarqubecloud

Copy link
Copy Markdown

@jmrplens
jmrplens merged commit dbd1b4a into main Oct 21, 2026
37 checks passed
@jmrplens
jmrplens deleted the issue-1132-dead-endpoint branch October 21, 2026 14:48
jmrplens added a commit that referenced this pull request Oct 22, 2026
… distinct-credential budget's reason from it (#1480)

Refs #954

The distinct-credential budget gave as its reason that GitLab
rate-limits failed authentication per source address, so a token sprayer
spends the deployment's standing with GitLab and the throttle GitLab
eventually applies lands on everyone using the server. GitLab's
documented failed-authentication ban covers Git and the container
registry, not the API, and whether an invalid-token REST request counts
against GitLab's per-IP limit on unauthenticated requests was an open
question (OQ-11, finding F-25). This is the first layer of the issue: I
measured the answer on GitLab 19.5 and rewrote the reason to it. The
keying change of the budgets is the next layer.

## The probe

I added
`TestAuthThrottle_InvalidTokens_SpendTheAddressUnauthenticatedAllowance`
to `test/e2e/gitlab/ce`. It reads the instance's unauthenticated API
throttle, turns it on at four requests in two minutes, and waits the
minute each GitLab process keeps its settings for. It then sends
invalid-token `GET /api/v4/user` requests itself, in `PRIVATE-TOKEN` as
the server's legacy verification sends a token, has the real binary in
HTTP mode verify an invented token, and sends one more invented token as
a Bearer token, as the OAuth verification sends one, reading what GitLab
counted off the `RateLimit-*` headers GitLab 19.5 writes on every
counted response. Once GitLab throttles the address, it sends an
anonymous request and the run's own token, starts three servers in HTTP
mode (one per surface) that verify the run's token and serve
`user.current`, and checks the address is still throttled afterwards. It
then waits out the window to see whether a ban follows, writes the
settings back, and checks nothing counts an invalid token any more.

GitLab 19.5 counts that throttle with Labkit rather than Rack::Attack.
Every Labkit cohort enforces by default
(`rate_limiter_use_labkit_rack_cohort_{1,2,3}_enforce`,
`default_enabled: true` at v19.5.0-ee), Rack::Attack then safelists
every request as `labkit_fully_enforced`, and every line of the run's
`api_json.log` says so. Labkit's counter is a Redis key that the first
request it counts creates with the period as its lifetime
(`lib/labkit/rate_limit/evaluator.rb` at labkit-ruby v5.5.0), so a
window opens with the first counted request and ends a period later,
while the `RateLimit-Reset` and `Retry-After` GitLab writes describe a
window aligned to the epoch (`RequestThrottleData.from_labkit_result`).
The probe therefore starts from a request counted as the first of its
window, and waits for a window to end a full period after an answer
counted in it (the answer that opened it, for the window it spends),
never by those headers. Its last verdict tells a ban (a 429 or a 403 a
full period after the window opened) apart from a count it did not
expect (a 401 counted as anything but the window's first request).
`TestThrottleWindow_OpensWithItsFirstCountedRequest` and
`TestNextWindowProblem_TellsABanFromAnUnexpectedCount` hold that model
with no GitLab, on the three calibrations of the recorded run.

The throttle is an instance-wide setting, and the issue decided that no
complete run may turn it on: a run cut short before the probe's cleanup
would leave it on for every package after it. So the probe runs only
when `E2E_AUTH_THROTTLE_PROBE=true`, only in Docker mode, and on its
own. `run-docker-e2e.sh` now refuses to start a complete run
(`E2E_GATE_SKIPS=true`) with that switch set, read from the environment,
the file `E2E_ENV_FILE` names or the repository `.env`, the same places
it reads the external network switch from, through a new
`complete_run_value` in `skip-gate.sh`. That lookup sources the file,
and the harness reads the same file with godotenv, which also takes
`KEY: value` and `KEY = value` and reads on past a reference to a
variable nobody set, so a switch written that way reached the probe
without the script seeing it. A complete run the script does not refuse
therefore starts with `E2E_AUTH_THROTTLE_PROBE=false` exported, and the
process environment outranks every file the harness reads.
`scripts/e2e_skip_gate_test.py` covers the lookup, the refusal, the
export and its place in the script. A complete run skips the probe, and
the skip gate fails a complete run on any skip nothing declares. So I
declared the skip in `cmd/audit_e2e_coverage/skip_declarations.go` under
a new category, `opt-in-probe`: a probe that changes an instance-wide
setting, which a complete run must never run. It is not a setting a run
left out, which the table still refuses to excuse. `test/e2e/README.md`
has a new section on the probe, and `cmd-utilities.md`, `testing.md`,
the header of `run-docker-e2e.sh` and the test-file counts in
`CLAUDE.md`, the repository map and the testing guide's tree follow the
table, the refusal and the export (the testing guide's tree also had
stale common and ee counts, now 175 and 48).

## What GitLab 19.5.0 answered

I ran the probe once, alone, on `gitlab/gitlab-ce:19.5.0-ce.0` (revision
4c9e34e1faa) on the Docker stack, and it passed in 425 s. Every request
from the test process and from the server children reached GitLab as the
same address, according to `api_json.log` and nginx's access log. That
run was of the probe's first version, before it modelled Labkit's window
and before it sent a Bearer token, so its `window_left` column is the
epoch-aligned `RateLimit-Reset` minus `Date`, which the probe no longer
reads. The probe as committed here has not run against GitLab. It needs
one run on its own before this merges, with the command in the README's
new section and `GITLAB_IMAGE=gitlab/gitlab-ce:19.5.0-ce.0`, and that
run's lines take the place of the record below.

```text
throttle as found: {Enabled:false Requests:3600 Period:3600}
off: invalid token 1..6: 401 (no throttle)          server refuses an invented token 401
on: calibration 1: 401 throttle_unauthenticated_api observed=1 remaining="3" window_left=51s
on: calibration 2: 401 throttle_unauthenticated_api observed=2 remaining="2" window_left=1m59s
on: calibration 3: 401 throttle_unauthenticated_api observed=1 remaining="3" window_left=1m58s
    server, HTTP mode, verifies an invented token: refused 401
on: invalid token after the server's verification: 401 throttle_unauthenticated_api observed=3
on: invalid token 1 toward the limit: 401 throttle_unauthenticated_api observed=4 remaining="0"
on: invalid token 2 toward the limit: 429 throttle_unauthenticated_api observed=5 retry_after=1m58s
on: anonymous request past the limit: 429 throttle_unauthenticated_api observed=6
on: the run's token past the limit: 200 (no throttle)
    three fresh servers verify the run's token and serve user.current: PASS on dynamic, meta, individual
on: invalid token after the servers were served: 429 throttle_unauthenticated_api observed=7
on: invalid token in the next window: 401 throttle_unauthenticated_api observed=1
restored: invalid token 1..6: 401 (no throttle)      server refuses an invented token 401
```

With the throttle off (as a self-managed instance ships it), nothing
counts an invalid token. With it on, the answer to OQ-11 is yes: every
request carrying an invalid token in `PRIVATE-TOKEN` is counted once
against the address, including the server's own legacy verification of
one. The source agrees: `Gitlab::Auth::RequestAuthenticator` rescues the
invalid token's authentication error to nil, so the request is
unauthenticated and the throttle keys on its address. Past the limit the
address loses its whole unauthenticated allowance, so an anonymous
request is answered 429 too. A token GitLab accepts is unaffected: it
was answered 200 and counted by no unauthenticated limit (GitLab's
authenticated API limit, which counts per user, was off, as it is by
default), and freshly started servers verified and served it throughout.
No ban followed: in the next window the invalid token was answered 401
and counted from 1 again.

The first version's record has no Bearer token in it, and the OAuth
verification sends one. GitLab 19.5 reads `PRIVATE-TOKEN` and a Bearer
token through the same finder
(`find_user_from_any_authentication_method`,
`lib/gitlab/auth/auth_finders.rb` at v19.5.0-ee), whose refusal
`RequestAuthenticator` rescues the same way, and GitLab.com, which runs
its throttle on, counted both alike when I asked it on 2026-10-21 at
01:34 UTC:

```text
GET https://gitlab.com/api/v4/user, one window, one address
PRIVATE-TOKEN glpat-<invented>   401 throttle_unauthenticated_api limit=500 observed=2
Bearer <invented, no prefix>     401 throttle_unauthenticated_api limit=500 observed=3
Bearer glpat-<invented>          401 throttle_unauthenticated_api limit=500 observed=4
no credential                    401 throttle_unauthenticated_api limit=500 observed=5
```

So the comment, the security page and the specification say what the
19.5 run observed for `PRIVATE-TOKEN`, and what GitLab.com and the
source say for a Bearer token, until the run above records it on 19.5
too.

Every count in the record is the probe's own. Calibration 2's count of 2
is calibration 1, sent 52 s earlier into the same Labkit window: the
first version of the probe read the epoch-aligned reset as the window's
end, waited it out and sent calibration 2 into the window calibration 1
had opened. The same reading made the run pass by timing. Calibration 3
opened the window the probe spent, which ended at 17:08:02.122, and the
wait for its end, the throttled 429's `Retry-After` plus a second
counted from a later 429, sent the next-window request at 17:08:10.489,
8.4 s after it. Started from a calibration with between 60 s and about
110 s left by the epoch, the same arithmetic would have sent it inside
the window and reported a ban that did not happen. That is what the
window model above fixes. Its counts and verdicts are the first
version's, and only the waits, the starting window and the one Bearer
request moved; the Bearer request is the fourth counted one of the
window, so the request past the limit is still the fifth.

## What lands on the server

The scenario cannot show what the server does with a token GitLab
throttles, because `Env.RefusedAdmission` fails a test whose credential
is admitted, and changing the harness is outside this layer. So I
checked it by hand on the same stack after the run. I turned the
throttle on again, spent the window with direct invalid tokens (429 from
the fifth on), started the staged binary in legacy HTTP mode, and posted
`initialize` with an invented token. GitLab answered the verification
429, which is no verdict on the token, and `verifyCredential` admits a
credential it got no verdict for, as it does when GitLab is unreachable.
The gate answered `initialize` 200 and the pool logged `created new
entry` with `scopes_detected: false`. The build took 16 s and five more
counted requests, because the entry's token and tier lookups retried
against 429s. In OAuth mode the same 429 is an upstream error and the
gate answers 503 without charging anything
(`TestOAuth_ThrottledUpstreamIsNotBlamedOnTheToken`).

So the rewritten reason says what was observed: a sprayer spends the
deployment's unauthenticated allowance with GitLab, not its users'
standing. What lands on the server once that allowance is spent is that
its verification of every token GitLab does not accept, every invented
one among them, gets no verdict: in legacy mode an invented token then
gets a pool entry, and in OAuth mode a 503. A token GitLab accepts is
still verified and served. The comment on `DistinctTokenBudget` and the
security page in English and Spanish say this and name 19.5, and the
note in `internal/config` and the paragraph in the tenant policy
specification beside F-25 say what GitLab answered. The comment no
longer calls the distinct count the earliest signal of that spending,
which it is not at the defaults: a sprayer at ten or more distinct
tokens a minute meets the failure lockout at its tenth, long before the
fiftieth. What the distinct count does is bound the sprayer the lockout
cannot, one kept under ten a minute, and block for longer each time it
is reached (a minute, ten, then an hour), where the lockout forgives
after a minute. The security page and the comment also give GitLab.com's
limits with their dates: in October 2026 it answered with 500
unauthenticated requests a minute per address, and GitLab announced 60
an hour for unauthenticated traffic from 19 October 2026, which its rate
limits page still lists as proposed and not in effect. The budgets'
defaults do not hold a sprayer below that 60, since the failure lockout
alone lets ten through a minute. F-25 stays on `AUB-001` and `AUB-003`
until the keying layer of this issue removes it.

I think the legacy admission under a throttled GitLab deserves an issue
of its own. While GitLab throttles the deployment's address, the legacy
door is open to every invented token: each one builds a pool entry (at
the pool's bound, evicting another credential's quiet entry, `POL-002`)
and holds a probe slot (`POL-006`) through the retries. The two doors
answering the same state of GitLab differently is a finding by the
specification's own rule, carried by the rows it concerns, and
`Validate` refuses a finding no issue carries, so the specification in
this layer records only what GitLab answered and leaves the two doors'
answers to that issue. Once it is filed, it becomes the next finding on
`ADM-001` and `ADM-002`, in the specification's findings table and in
the paragraph beside F-25, with the keying layer's register change or
before it. This layer changes no server code.

## The package the skip declaration touches

The declaration puts this layer in `cmd/audit_e2e_coverage`, so I
measured the whole package. On the parent, exported in full (a `git
archive` leaves out `test/` and `docs/`, which the package's own tests
read), `make coverage-mutants PKG=./cmd/audit_e2e_coverage` reported
Killed 671, Lived 3, Not covered 88, the same as this branch before the
third commit. The three that lived were real gaps, both in code this
branch does not otherwise touch: `report()` of the fine-grained fold
behaved the same under `||` as under `&&`, since no test had sessions
without calls or calls without a session line, and `enclosing`'s
half-open span was never asked about at its edges, which no site the
scanner reads can reach. The third commit asserts both directly, and the
package now reports Killed 674, Lived 0, Not covered 88, on the final
head too, with the same 88 positions. None of the 88 has a statement
counter to be covered by: they sit in `switch { case ... }` expressions
and package-level initializers, the tool artifact
`docs/development/testing/README.md` describes, so I applied each one by
hand to a copy of the tree and ran the package's tests against it. 53
die: 50 under `go test` as it runs, and three more whose `||` turned
`&&` makes a condition that cannot hold, which go vet refuses and the
suite fails with vet off as well. The other 35 turn a `+` between two
string literals into a `-`, which does not compile: 29 in `portmap.go`,
four in `runtime_record.go` and two in `static.go`. gobco reports 1318
of 1318 conditions.

## How I tested it

- The probe's own unit tests, with no GitLab:
`TestThrottleAnswer_ReadsTheRateLimitHeaders`,
`TestThrottleWindow_OpensWithItsFirstCountedRequest` and
`TestNextWindowProblem_TellsABanFromAnUnexpectedCount`. Against the
probe as it stood before the window model they do not build (`answer.At
undefined`, `undefined: opensWindow`, `undefined: windowMargin`,
`undefined: nextWindowProblem`); on this branch all three pass with
their twelve subtests.
- The probe itself, once on GitLab 19.5.0 CE in its first version, as
recorded above. The version committed here still needs its one run.
- The refusal and the export: `scripts/e2e_skip_gate_test.py` covers the
lookup, the refusal, a run of the lifecycle script with the switch in
the environment and in the named file, and a complete run started with
the switch written as `KEY: value`, as `KEY = value`, after a reference
to an unset variable, and not at all, whose stand-in `docker` records
the switch each call inherits. Against the parent's scripts the 33 tests
end with 14 failures and 3 errors, those four cases among them, seeing
the switch unset rather than `false`, and the placement check finding no
export; on this branch all 33 pass, among them a run not asked to be
complete, which keeps the caller's `true`. shellcheck is clean on both
scripts.
- The skip a complete run sees, recorded against the same stack without
the switch and fed to `audit_e2e_coverage -check-skips -runtime ce`: on
the parent, `1 skipped on ce, 0 declared, 1 undeclared` and exit 1; on
this branch, `declared (opt-in-probe)`, `1 skipped on ce, 1 declared, 0
undeclared, 0 stale declarations` and exit 0.
- `go test -tags e2e -c` of `./test/e2e/gitlab/...`, `go vet` and
`golangci-lint --build-tags e2e` on the CE package,
`cmd/audit_e2e_coverage`, `internal/serverpool` and `internal/config`,
all clean.
- Statement coverage of `cmd/audit_e2e_coverage`, `internal/serverpool`
and `internal/config` is 100%. The last two change only in comments, and
I measured them anyway: gobco 418 of 418 and 424 of 424; gremlins Killed
215, Lived 0, Not covered 6 on `internal/serverpool` and Killed 205,
Lived 0, Not covered 9 on `internal/config`. Applied by hand, all nine
of `internal/config`'s not covered die and four of
`internal/serverpool`'s six, the other two being a `+` between string
literals in `token.go`.
- `make check-tenancy`: 87 rows, 0 findings. `make
tenancy-code-identity`: against the plain parent, 7 sections differ
besides the line table, the documented case of a change that inserts
lines into the server's source, here comment lines in
`distinct_token_budget.go` (30) and `config.go` (3). Against the parent
with blank comment lines inserted at the same positions, whose
non-comment lines are the parent's at this branch's line numbers, the
two binaries are identical in every allocated section besides
`.gopclntab`.
- `make check-e2e-static`, `check-test-file-names`,
`check-test-subtests`, `check-test-goroutines`, `check-dead-consts`,
`check-md-tables` and `check-doc-tool-names` pass. markdownlint on the
changed pages, the em dash check over the branch and
`scripts/check-doc-links.mjs` are clean. The site's `i18n:check`,
`facts:check`, `chips:check`, `llms:check`, `build` and `lint` pass.
- `gen_testing_docs --check` reports the generated counts in
`testing.md` behind by the test functions and files this branch adds.
That is the committed artifact this stack regenerates once at its top.
- I rebased the stack onto `main` at `a0e4e6cf6`, with no conflict and
every commit's patch unchanged. The figures above were measured on the
branch as it stood on `19746966465`. Of the packages they measure, only
`internal/serverpool` moved on `main` since, in the place of a doc
comment and in its tests (#1472); `main` also changed files under
`test/e2e` and `docs/` that the tests of `cmd/audit_e2e_coverage` read,
which is why I ran those tests again here. At this head `go build
./...`, `go vet`, `golangci-lint --build-tags e2e` and `go test` on
`cmd/audit_e2e_coverage`, `internal/config` and `internal/serverpool`
pass, each at 100.0% of statements, the e2e suite compiles, the skip
gate's 33 Python tests pass, `make check-tenancy` reports 87 rows and 0
findings, and `make check-closed-servers` and `make check-godocs`, the
two gates `main` gained since (#1474, #1478), pass.
jmrplens added a commit that referenced this pull request Oct 22, 2026
…e /48 and name the unix-socket peer (#1481)

Closes #954.

This is the second layer of #954. The first corrected the
distinct-credential budget's stated reason from what GitLab 19.5
answered; this one changes the key every authentication budget before
admission is charged to, which is what the issue opened with, and adds
the bound one level up that the decision of 2026-09-28 asked for, with
the constants and the /64 the decision of 2026-10-17 fixed.

## What changed

The budgets were keyed on the whole client address. An IPv6 caller holds
at least a /64 and can use any address in it, so rotating the low bits
handed it a fresh fast budget and a fresh distinct-credential budget on
every request, and nothing short of the coarse transport-source budget
(500 distinct keys a minute, and only behind a trusted proxy header)
ever cut it off. Every caller on a unix socket, meanwhile, shared
whatever `RemoteAddr` spelled there, and `--trusted-proxy-header` could
not separate them, because a header was believed only from a peer that
parses as an address.

I added `budgetKey` in `cmd/server/client_ip.go`: a whole IPv4 address,
an IPv4-mapped address as the IPv4 address it names, and an IPv6 address
by the fixed /64 it lies in. Both doors read the fast budget's and the
distinct-credential budget's key through it, and the transport source
(`AUB-002`) is keyed the same way, so a proxy reaching the server over
IPv6 is one source however it rotates its own low bits.

A caller holding a /48 holds 65536 /64s, so I added the aggregate the
decision asked for as register row `AUB-006`: the /64s of one IPv6 /48
that had a credential refused, counted on the distinct-token budget's
machinery with the /64 as the distinct value (held as a digest, in a
table `AUB-004` caps with the other two), in `AUB-003`'s window and up
`AUB-003`'s ladder, at `tenancy.AuthNetworkFailingPrefixes = 10`, a
register constant no flag sets. I read "failing /64s" as /64s that had a
credential refused, the same thing the distinct-credential budget
counts, so a /64 whose clients only forgot their header does not move
it. Its windows fall back to their defaults when a zero switches
`AUB-001` or `AUB-003` off, so turning those off does not turn it off
(`INV-015`); an IPv4 address and a unix-socket peer have no /48 and are
not counted.

What the fixed prefixes cost is stated rather than assumed away. Callers
that share a /64 share its budgets, and on IPv6 that is more than the
IPv4 NAT comparison brings to mind: the hosts of one IPv6 subnet, which
on private IPv4 each had a key of their own, the customers a provider
numbers from one shared /64, and every link-local caller, since the zone
is dropped and fe80::/64 is the same prefix on every interface. A
stateful NAT64 gateway changes nothing, since the server sees the
gateway's IPv4 address and keys it whole, but a stateless translator in
front of an IPv6-only server (SIIT-DC, RFC 7755) writes every IPv4
client into its RFC 6052 prefix, so under `64:ff9b::/96`, or any prefix
of /64 or longer, they all share one /64, and under any prefix of /48 or
longer one /48, where each had a key of its own before. The aggregate
counts a refused credential whoever presents it, so an organization
whose whole network is one /48 reaches its ten when callers on ten of
its subnets present an expired or revoked token inside the
distinct-credential window, and the /48 is then refused for a minute,
then ten, then an hour, for every credential the deployment does not
already serve; in legacy mode that is every token outside the pool,
which `--max-http-clients` bounds at 100 by default. A /48 is not always
one organization's either: a provider that delegates each subscriber a
/56 or a /60 puts 256 or 4096 of them in one, and a mobile network gives
each device a /64 of its own (RFC 6459), so ten unrelated subscribers
presenting an expired token inside the window block the /48 for every
other one, and no flag turns the aggregate off. The security page says
all of this. The ten is the decision's figure and a register constant,
so I left it as it is.

A connection accepted on a unix socket is now the one key `unix`, read
off the listener's address on the request context rather than off
`RemoteAddr`, which is `@` on Linux, empty or a path elsewhere. `unix`
is accepted in `--trusted-proxies` and trusts the socket's peer and no
address, so the proxy on a socket names each caller exactly as a proxy
on loopback TCP does. The host guard asks the same question through the
same set (`trustedProxies.trustsPeer`), so the two cannot read the list
differently.

Telemetry counts the aggregate's refusals under a fourth reason,
`network_aggregate`, since `authBlockCounters` drops a reason it does
not know. `longestAuthBlock` now takes the four budgets as a list in the
order that breaks a tie.

## Release note: a breaking startup check, and budgets shared by /64

Two pairings of the proxy flags with `--http-addr` now refuse startup,
because each trusts nobody while reading as if it trusted someone. A
unix-socket `--http-addr` whose `--trusted-proxies` names addresses and
no `unix` used to start with `--trusted-proxy-header` silently ignored,
so every caller on the socket shared one budget; it now stops with
`--trusted-proxies does not name unix and --http-addr ...`, and the fix
is to add `unix` to the list. `unix` in `--trusted-proxies` on a TCP
`--http-addr` is refused as well. A socket listener with neither proxy
flag starts as before.

The budgets also share differently. Every address of one IPv6 /64 now
spends one fast budget and one distinct-credential budget, so the hosts
of one IPv6 subnet share them as callers behind one IPv4 NAT share an
address's, the IPv4 clients a stateless translator (SIIT-DC) writes into
`64:ff9b::/96` all share one /64, and ten /64s of one /48 that had a
credential refused inside `--auth-distinct-token-window` block the whole
/48 for every credential the deployment does not already serve. A
credential the deployment already serves is still exempt from every
block.

## The register and the documentation

`IDN-004` and `IDN-005` derive at `budgetKey`, `AUB-001` to `AUB-003`
record issue 954 and carry F-14 and F-25 no longer (F-34 stays for
#958), the key vocabulary gains `network` with its mint cost and
evidence, the address and source evidence state the /64 and the unix
key, `AUB-004` names the /48 table among those it caps, and `AUB-006` is
the eighty-eighth row with its frozen pin and its entry in
`specRequirementIDs`. The specification's key table and findings say the
same, with what the /64 and the /48 cost. I did not declare the two
startup refusals as register refusals, as none of
`validateTrustedProxyConfig`'s existing refusals is declared either.

The site pages change in English and Spanish together: the security
page's budget table gains the aggregate, with its fixed ten and its
configurable window, its introduction names the transport-source budget
a trusted proxy header adds, and a paragraph says what an address is,
who shares one (a stateless translator's IPv4 clients among them) and
what the /48 costs, to an organization whose network is one and to the
unrelated subscribers a provider places in one; the TLS page's
unix-socket section, which told a shared deployment to give the socket
up, now shows `--trusted-proxies=unix` and the two refused pairings, and
the remote deployment page's socket unit and its table of shapes carry
it with the header it requires; the HTTP server, CLI, configuration and
environment references name `unix` and the /64 for both address budgets,
and the CLI reference says the /48 aggregate counts in the
distinct-credential window; wherever a page said the header is believed
only from a listed address it now adds the socket's peer when `unix` is
listed; the troubleshooting rows for a 429 name the /48 and `unix`; the
enterprise overview's card for the TLS page describes what that page now
says; the telemetry page names `network_aggregate`.
`docs/development/configuration.md`, `http-invariants.md` and the
copilot instructions follow; the flag usage, the curated help and its
ENVIRONMENT block name the /64 for both address budgets, and so do
CLAUDE.md's two rows for them. The refusal of a `--trusted-proxy-header`
with no list names `unix` beside addresses in its remedy, since on a
socket listener a list of addresses alone is the pairing the next
startup refuses.

## Validation checklist

`AUB-006` is the new row; `AUB-001` to `AUB-003` keep every answer below
but the reading of their key, which VAL-002 covers.

- VAL-001, question: admit, like every budget before admission.
- VAL-002, key: `network`, new in the vocabulary
(`internal/tenancy/key.go` and the specification's key table): the IPv6
/48 a charged client address lies in, read by `networkKey`, minted as an
address is, with its evidence that another value is another assignment
while a provider may give one end site many /64s inside one /48, and may
put many unrelated subscribers inside one. `AUB-001` and `AUB-003` stay
on `address` and `AUB-002` on `source`, whose evidence now states the
/64 and the `unix` key; neither mint cost changes.
- VAL-003, class: A, before admission where no tenant exists yet, like
the other budgets.
- VAL-004, ceiling or share: a budget, a ceiling on what one /48 may
spend that promises nobody a share.
- VAL-005, process partner: it protects no process resource; its table
is one of the three `AUB-004` caps.
- VAL-006, at capacity: the /48 table is a `DistinctTokenBudget` capped
at `AuthTrackedSources` (4096) keys, and like the distinct-credential
table it stops counting a new key once full rather than evicting a
record, since the records there are the evidence (`AUB-004`'s second
refusal).
- VAL-007, refusal: the gate's, at both doors: `429`, `-42900`,
`Retry-After` the longest block holding the request and never under a
second, no challenge, retry-later class, in every budget's words (`Too
many failed authentication attempts from this address. Retry later with
a valid token.`); a credential the deployment already serves is exempt,
as from every budget. On stdio: none.
- VAL-008, charges: a credential GitLab refused, at both doors and from
the OAuth door's memory of a refusal too, and nothing else: not a
request that carried no credential, and none of the failures the
register declares uncharged.
`TestDecisions_NetworkAggregate_IsTheConstantIssue954Added` holds that
failure by failure against the register's `Failures`, and `make
check-tenancy` (G7) holds the code to it.
- VAL-009, value: `tenancy.AuthNetworkFailingPrefixes = 10`, `Source:
Constant`, pinned in `TestValues_HoldTheirPins`; no flag or variable, so
no bounds of its own and no malformed-value policy; zero is not
applicable. Its window is `AUB-003`'s and its ladder's step `AUB-001`'s,
each its default when the operator's is zero.
- VAL-010, telemetry: `gitlab_mcp.auth.blocks` counts its refusals under
the new reason `network_aggregate`, the reason being the metric's only
dimension and the address not one (`INV-013`). The refusal's log line
names the budget beside `ip` and `source`, as it already did for the
other three, throttled like every refusal a caller can cause.
- VAL-011, stdio: not applied (`StdioKey` none); stdio has no door.
- VAL-012, measurement: not applicable. The aggregate does not claim to
protect one population from another; it bounds what one /48 may spend,
and its cost to an honest population sharing one is stated above and on
the security page rather than claimed away. `make bench-fairness`
measures four bounds, none of them an authentication budget.

## Where this departs from the plan

CLAUDE.md changes in two rows only, the two address budgets', which now
name the /64; it has no row for `--trusted-proxies` (its table lists
variables, and this flag has none), so the flag's new entry is described
in `docs/development/configuration.md` and in the flag help.
`internal/serverpool` is unchanged: the aggregate reuses
`DistinctTokenBudget` rather than adding a type beside it.
`cmd/audit_tenancy` needed no new exemption.
`docs/development/testing/testing.md` is not regenerated here, as the
stack's top regenerates once, so `gen_testing_docs --check` reports this
layer's renamed and added tests until then.

## How I tested it

Tests first. On the parent the new unit tests do not build (`undefined:
KeyNetwork`, `undefined: AuthNetworkFailingPrefixes`, `unknown field
NetworkAggregate`, `undefined: mcpotel.AuthBlockNetworkAggregate`). I
then ran the four new transport tests from `test/e2e/http` against the
parent's binary, and all four fail: `forty addresses rotating inside one
/64 were never cut off`, `a /64 the /48 never used = 401 once 10 of its
/64s had a credential refused, want 429`, `the server never became
healthy` for a socket with `--trusted-proxies=unix`, and neither refused
pairing stopping with a refusal that names both flags (the socket one
started). On this branch they pass at both doors, beside
`TestGate_FailureBudgetIsPerAddress` and
`TestGate_RotatingProxyHeader_IsBoundedByTheTransportSource` unchanged.

Review added three unit tests, each red on the parent with the one
helper it needs copied in: the bearer guard's twin of the gate's
unix-socket test (on the parent, with `unix` trusted, the second
caller's valid token is answered 429 and never verified), the host
guard's test asked through `permits` on a guard bound to a host of its
own (on the parent, a unix-socket connection with `unix` listed has its
undeclared Host refused), and a `main` row for the socket pairing (the
parent tries to bind the path and stops with `its directory is not
usable`, naming neither flag). The rewrite of
`TestListener_UnixSocket_SharesOneAuthenticationBudget` sits in the
commit that adds the startup refusal it would trip, so the transport
module passes with that commit's own binary as well as at the tip.

The unit tests cover `budgetKey` (whole IPv4, IPv6 /64, mapped, zone,
`unix`), `networkKey`, the unix-socket peer on three spellings of
`RemoteAddr`, `unix` in `--trusted-proxies` and `trustsPeer`, the host
guard's exception through `permits`, rotation inside one /64 spending
one fast budget and one distinct-credential record at both doors, ten
/64s of one /48 tripping the aggregate and nine not, a credential-less
/64 and an IPv4 /24 never moving it, both unix-socket cases at both
doors, the aggregate staying on with every other budget off and counting
in the configured windows (synctest), the four-budget
`longestAuthBlock`, the telemetry count, both startup refusals through
the validator, through `main` and on the binary, and the register pins.

Results: `go test ./cmd/server ./internal/tenancy ./internal/mcpotel
./internal/config` pass; statement coverage 100% on all four; gobco
2300/2300 on `cmd/server`, 450/450 on `internal/tenancy`, 288/288 on
`internal/mcpotel`, 424/424 on `internal/config`; `make
coverage-mutants` killed 145 with 0 lived and 0 not covered on
`internal/mcpotel`, killed 252 with 0 lived on `internal/tenancy` (its
65 not covered are the `N * time.Minute` initializers of `values.go`,
the constant-initializer artifact `docs/development/testing/README.md`
describes, none of them a line this changes), and killed 230 with 0
lived on `cmd/server` restricted with `--exclude-files` to five of the
six non-test files this changes there (`auth_blocks.go`, `auth_gate.go`,
`bearer_guard.go`, `client_ip.go` and `host_guard.go`; a whole-package
run costs the 44-second suite per mutant), whose 8 not covered are the
same artifact in constants of `auth_gate.go` this does not touch.
`main.go`, the sixth, is left out for its size; its one new condition,
the `validateTrustedProxyListener` check in `runHTTP`, negated by hand
fails seven tests,
`TestMain_HTTPFlags_EachRefusalNamesTheFlagThatCarriedTheValue` and
`TestRunHTTP_HandsTheToolLayerItsSettings` among them, and the rest of
what it changes is flag help and the wiring of the new budget, which
`TestRegisterMCPHandlers_SweepEachExpiringTableTheyBuild` and the
transport tests exercise; `make check-tenancy` 88 rows, 0 findings;
`make check-e2e-static`, `check-test-goroutines`, `check-test-subtests`,
`check-test-file-names`, `check-dead-consts`, `check-doc-tool-names` and
`check-action-ids` pass; `make test-e2e-http` 678 tests at the tip and
672 with the feat commit's own binary, the 8 Docker-backed ones skipped
where no daemon runs; `make test-e2e-stdio` 234 tests; `golangci-lint`
clean on the touched packages under the `e2e`, `httpe2e` and `stdioe2e`
tags; `go test -tags e2e -c` compiles `test/e2e/gitlab/...`; the site's
`i18n:check`, `facts:check`, `chips:check`, `llms:check`, `build` and
`lint`, `scripts/check-doc-links.mjs`, `format_md_tables --check`,
markdownlint and the em dash check pass.

I then rebased the stack onto `main` at `a0e4e6cf6`, with no conflict
and every commit's patch unchanged, so the results above were measured
on the branch as it stood on `19746966465`. At this head `go build
./...`, `go vet`, `golangci-lint --build-tags e2e` and `go test` on the
four packages pass, each at 100.0% of statements; `make check-tenancy`
reports 88 rows and 0 findings; `make test-e2e-http` passes; `make
test-e2e-stdio` passes with 245 tests, the eleven more than above being
the ones `main` added to `elicitation_declared_version_test.go` (#1469);
and `make check-closed-servers` and `make check-godocs`, the two gates
`main` gained since (#1474, #1478), pass. Of the packages this layer
touches only `cmd/server` moved on `main`, in two test files (#1472), so
the gobco figures, which count the conditions of the package's code,
stand; I did not repeat the mutation runs.

The last commit came down from the layer above once CI ran this one
alone: `TestRunMain_TheTree_PassesTheGate` and
`TestProductionRegister_IsTheRegisterAsItIs` in `cmd/audit_tenancy` pin
what the gate reads off the real register, and `AUB-006` moves two of
those figures (95 refusals rather than 93, and a derivation of the
`network` key between the transport source and the refused credential),
so both failed at this layer's head until the commit that updates them
sat here.
jmrplens added a commit that referenced this pull request Oct 22, 2026
…1482)

Closes #982.

This is the third layer of the authentication-budget stack, on top of
the /64 key of #954. It bounds the record the transport-source budget
keeps of the pairs it charged, as the decisions of 2026-09-27 and
2026-10-17 on the issue set out, and moves `AUB-002` off F-35.

## What was wrong

Behind `--trusted-proxy-header` the transport-source budget (`AUB-002`)
charges a transport source once per distinct failing primary key inside
its window, and to know which keys it already charged it keeps a map of
(source, key) pairs, `transportBudget.charged`. Both halves are the
caller's to choose, and nothing but the five-minute sweep removed a
pair. A blocked source adds none, but the limiter under the budget
records at most 4096 sources (`AUB-004`) and at that cap stops recording
new ones. A source arriving then is never blocked by this budget, and
the budget went on keeping a pair for every key it failed with.

## What changed

`AuthRateLimiter.RecordFailure` now reports whether it counted the
failure: true for a new key with room, for a live record counted into
and for a lapsed record restarted in place, false for the one failure
the cap leaves uncounted, a new key the full table has no room for.
`AuthRateLimiter.Limit` returns the count the limiter blocks at. Nothing
else reads either.

`transportBudget.charge` keeps a pair only when its limiter recorded the
failure, so a source the full limiter never recorded keeps no pair at
all. That also removes a quieter defect: the pair kept for an unrecorded
failure stood in for a charge that never happened, so the same key
failing again once the limiter had room was deduplicated against it and
went uncounted. It is now charged as a first failure, which `INV-007`
allows, since the caller caused it. I moved the limiter call under the
budget's own lock, so two concurrent failures of one fresh pair are
still charged once, as they were when the pair was stored before the
limiter was asked.

A source holds at most its limiter's limit in pairs (500 in production,
`tenancy.TransportSourceDistinctKeys`). At that bound a fresh key takes
the place of the source's own oldest pair and is charged as before, as
decided on the issue on 2026-10-17. Stopping at the bound instead would
let a source just released from its block fail with fresh keys uncharged
while the pairs that blocked it are still held, and reclaiming only the
source's own pair never takes another source's (`INV-005`). A key whose
pair lapsed is re-opened in place and takes no room. I read the bound
off the limiter rather than off the constant so that it is the count the
limiter blocks at for every limiter, the tests' small ones included; in
production the two are the same value.

Keeping a pair only for a recorded failure does not bound the sources by
itself, although the decision of 2026-10-17 reads as though it did. The
limiter's full table lets a lapsed record go at most an eighth of a
window after it lapses and takes a new source in its place, while the
budget forgot a pair only at its five-minute sweep (`AUB-005`), so a new
table's worth of sources stayed for every window between two sweeps:
four tables' worth at the default window, measured below, and more the
shorter the window, since `AUB-001` accepts any positive one. So once
the budget indexes as many sources as the limiter's table can hold, a
new source first makes it forget every source none of whose pairs is
live, at most once every eighth of a window, as the limiter sweeps its
own full table. Past that many, the sources it holds were all charged
within one window and an eighth, each while the limiter held a record of
it, and a record stays in that table for a window at least, so they are
at most three times what the table holds (12288 sources of at most 500
pairs each), whatever the window. That is the decided bound times three
rather than the decided bound itself: holding it to one table exactly
would cost a pass over the index for every new source at the ceiling,
which is the amplifier the limiter's own sweep is throttled against. The
pass reads each source's last charge rather than its pairs, and the
pairs of a source it forgets are deleted once.

The flat map keeps its key format, the source and the key joined by a
NUL, so the tests that read it directly are untouched. Beside it a
per-source index of the pairs each source holds and when it was last
charged bounds the source's share, finds its oldest pair, and finds the
sources with no live pair. Finding the oldest scans at most 500 entries
and builds no string, and only a source at its bound pays for it: one
never blocked whose lapsed pairs the sweep has not reached yet, one
whose block has lifted while the pairs that earned it are still held,
live or lapsed and not yet swept, or one charged by a request that
passed the gates before the block. The sweep walks the index, drops each
lapsed pair from both, and drops a source with its last pair.

The comments on `transportFailureLimit`, `transportBudget`, `charge`,
`maxTrackedAuthSources`, `AuthRateLimiter`, `RecordFailure` and `Limit`
now say what is bounded and what is not, instead of claiming the map
cannot grow because a blocked source stops charging.

## The register and the specification

`AUB-002` records issue 982 among its decisions, declares `AtCapacity:
ReclaimOwnThenRefuse` (the source's own oldest pair for a fresh key, and
no place for a source the limiter did not record), names
`transportBudget.keepLocked`,
`transportBudget.forgetLapsedSourcesLocked` and
`transportBudget.reclaimOldestLocked` among its enforcing sites, and
carries F-16 alone. `AUB-004` declares `transportTrackedSources`, the
budget's alias of its value, which is the index size at which the budget
forgets the sources with no live pair. F-35 stays in the list, filed as
issue 982's, and joins the answered set
`TestDecisions_AFindingNoRowCarries_IsAnsweredByItsIssue` names. The
specification's F-35 paragraph now says how it was answered, both bounds
and the factor of three included, and the count of answered findings is
fifteen.

What this leaves alone is the saturation itself: a source arriving once
the limiter's table is full is still blocked by neither table. As the
issue's second decision says, that belongs to the /48 aggregate of #954
and the capacity rules of #958.

`cmd/audit_tenancy/declarations.go` needed no change: the one new
package-level value is the declared alias, `RecordFailure` is still
called only from the declared charge helpers, and `make check-tenancy`
reports 0 findings over 88 rows and 370 declared sites.

Validation checklist for the change to `AUB-002`: VAL-001 to VAL-005,
VAL-007, VAL-009 and VAL-011 are unchanged (Admit, keyed on the
transport source, class A, a budget rather than a share, no process
partner, the same 429 refusal, the constant 500 whose zero no operator
sets, and nothing on stdio, where the budget does not exist). VAL-006 is
the change: the row now states its capacity behavior and still marks the
structure as a table on caller-chosen values, and the index it keeps is
bounded across sources by `AUB-004`'s value as above. VAL-008: what is
charged is still a judged failure, a credential GitLab refused or one
that was missing, and two such failures are now charged where the parent
deduplicated them, both the caller's own doing (`INV-007`): the next
failure of a key whose earlier failure the full limiter did not record,
and a fresh failure of a key whose pair was reclaimed at the source's
bound while it was still live. Forgetting a source drops no live pair,
so it charges nothing new. VAL-010: telemetry is unchanged, the refusal
is still counted under `transport_source` and nothing about a source or
a key is recorded. VAL-012: the change claims a memory bound rather than
protection of one population from another, so the measurements are the
issue's own and the rotation the cross-source bound answers, below.

## How it was tested

The tests came first. Eleven new tests in `cmd/server/auth_gate_test.go`
pin the bounds, and nine of them read only the flat map, so they compile
against the parent as it stood before the stack was rebased onto `main`
(`c1835d5b2`, now `2dbd166fd`) and fail there:

-
`TestTransportBudget_Charge_KeepsNoPairForASourceTheFullLimiterDidNotRecord`:
4096 sources fill the limiter, then one more fails with 100000 keys.
Parent: 100000 pairs for it, 104096 in all. Now: none, 4096 in all, and
the source is not blocked.
-
`TestTransportBudget_Charge_BlocksAtTheLimitAndKeepsNoMoreThanItsPairs`:
blocked at 500 keys as before, then 500 more. Parent: 1000 pairs. Now:
500, and another source's pair is kept.
- `TestTransportBudget_Charge_AfterTheBlockLifts_StillChargesAFreshKey`
(synctest): after the block lifts with one of its two pairs still live,
a fresh key reclaims the source's own oldest pair and the next one
blocks it again. Parent: 3 and then 4 pairs.
- `TestTransportBudget_Charge_ALapsedPairReopensInPlace` (synctest):
re-opening a lapsed pair keeps both pairs, and the next fresh key takes
the oldest. Parent: 3 pairs.
- `TestTransportBudget_Charge_ChargesAFailureOnceTheFullLimiterHasRoom`
(synctest): an unrecorded failure keeps no pair and blocks nothing, and
the same key is charged once the limiter has room. Parent: the pair is
kept and the later failure goes uncharged.
- `TestTransportBudget_Cleanup_KeepsTheRoomOfAPairItKeeps` (synctest): a
pair the sweep keeps still counts against its source's room. Parent: 3
pairs.
- `TestTransportBudget_Charge_TurningTheLimiterOverDoesNotStackSources`
(synctest): four generations of 4096 new sources, each arriving once the
last one's records and pairs have lapsed. Parent: 8192 sources after the
second generation. Now: 4096 after each.
- `TestTransportBudget_Charge_HoldsAtMostThreeTablesOfSources`
(synctest): six generations, each source charged its last key a second
before its record lapses, so its pair outlives the record. Parent: 16384
sources after the fourth generation, past the 12288 of three tables.
Now: at most 8192.
-
`TestTransportBudget_Charge_ForgetsLapsedSourcesAnEighthOfAWindowApart`
(synctest): a new source at the ceiling forgets the lapsed sources, the
next one an eighth of a window less a nanosecond later does not, the one
exactly an eighth later does, and a source last charged exactly a window
ago is kept. Parent: 4097 sources where the first sweep leaves 4.

The tenth, `TestTransportBudget_Cleanup_ForgetsASourceWithNoPairLeft`,
reads the per-source index this change adds, so it cannot compile
against the parent; it pins that the sweep leaves no empty source
behind. The eleventh,
`TestTransportBudget_Charge_ChargesAConcurrentFreshPairOnce`, pins what
asking the limiter under the budget's lock buys: 32 concurrent failures
of one fresh pair, 200 times, block no source whose limit is two. It
passes on the parent, which stored the pair before it asked the limiter,
and failed five runs out of five (14 to 26 sources of 200 blocked) on a
copy of this branch that asks the limiter after releasing the lock. In
`internal/serverpool`,
`TestAuthRateLimiter_RecordFailure_ReportsWhetherItRecorded` and
`TestAuthRateLimiter_Limit_IsTheCountThatBlocks` fail to compile against
the parent (`RecordFailure` returned nothing and `Limit` did not exist).
In `internal/tenancy`,
`TestDecisions_TransportSourcePairs_AreBoundedByIssue982` and the
answered-set expectation of
`TestDecisions_AFindingNoRowCarries_IsAnsweredByItsIssue` fail against
the parent's register. All of them pass on this branch, and
`TestTransportBudget_ChargesOncePerKeyPerWindow` and
`TestTransportBudget_Cleanup_ForgetsLapsedPairsOnly` pass unchanged.

The issue's measurement, run with a throwaway test against the parent
and against this branch (never committed). With the limiter's table
full, one more source failing with N distinct keys:

| N      | Map length, parent | Map length, this branch |
| ------ | ------------------ | ----------------------- |
| 10     | 4106               | 4096                    |
| 1000   | 5096               | 4096                    |
| 10000  | 14096              | 4096                    |
| 100000 | 104096             | 4096                    |

With the table not full, the source is blocked after 500 distinct keys
on both, and the map holds N pairs on the parent (1000, 10000 and
100000) and 500 on this branch.

The rotation the cross-source bound answers, measured the same way with
a limit of 10 so that the pairs stay small, and inside one sweep
interval, since the five-minute sweep resets both. Each generation is
4096 new sources failing with 10 keys each: in the first rotation a
generation arrives a window and an eighth and a second after the last,
in the second each source fails with its last key a second before its
record lapses and the next generation arrives two seconds later. Keeping
a pair only for a recorded failure, without forgetting sources, measures
exactly as the parent does here.

| Generation | Rotation, parent | Rotation, this branch | Late last key,
parent | Late last key, this branch |
| ---------- | ---------------- | --------------------- |
--------------------- | -------------------------- |
| 0 | 40960 | 40960 | 40960 | 40960 |
| 1 | 81920 | 40960 | 81920 | 81920 |
| 2 | 122880 | 40960 | 122880 | 81920 |
| 3 | 163840 | 40960 | 163840 | 81920 |

Checks, all on this branch:

- `go test ./cmd/server ./internal/serverpool ./internal/tenancy
./cmd/audit_tenancy -count=1`: pass, each at 100.0% of statements; `go
test -race` over the budget, gate and bearer guard tests: pass;
`cmd/bench_resources` and `internal/finegrained`, the other readers of
the register and the pool: pass.
- `make coverage-conditions` (gobco, `GOBCO_GATE=all`): `cmd/server`
2316/2316, `internal/serverpool` 418/418, `internal/tenancy` 450/450.
- `make coverage-mutants` on the files this changes, the others excluded
with `GREMLINS_FLAGS=--exclude-files`: `cmd/server/auth_gate.go` 112
killed, 0 lived, and 8 not covered, which are mutations in package-level
constant initializers (the two negative error codes and the string
concatenations of the two missing-token messages) that carry no
statement counter, the tool artifact
`docs/development/testing/README.md` describes, and the parent reports
the same way; `internal/serverpool/rate_limit.go` 22 killed, 0 lived, 0
not covered; `internal/tenancy/decisions_admit.go` has no mutable
expression.
- `make check-tenancy`: 0 findings.
- `make test-e2e-http`: 678 tests, 8 skipped for want of Docker, and
`TestGate_RotatingProxyHeader_IsBoundedByTheTransportSource` and
`TestGate_BlockedAddressStillServesAnAdmittedCredential` pass unedited.
- `golangci-lint run --build-tags e2e` and `golangci-lint fmt --diff` on
the four changed packages, the e2e compile, `check-test-subtests`,
`check-test-goroutines`, `check-test-file-names`, markdownlint on the
specification, `format_md_tables --check`, `check-doc-links` and `make
check-em-dash`: clean.

I then rebased the stack onto `main` at `a0e4e6cf6`, with no conflict
and every commit's patch unchanged, so the checks above were run on the
branch as it stood on `19746966465`. Of the packages this layer touches,
`main` moved `cmd/server` in two test files and `internal/serverpool` in
its tests and the place of one doc comment (#1472) and
`cmd/audit_tenancy` in its scaffolding (#1479). At this head `go build
./...`, `go vet`, `golangci-lint --build-tags e2e` and `go test` on the
four packages pass, each still at 100.0% of statements; `make
check-tenancy` still reports 0 findings over 88 rows and 370 declared
sites; and `make check-closed-servers` and `make check-godocs`, the two
gates `main` gained since (#1474, #1478), pass. The gobco figures count
the conditions of code `main` did not change, so they stand; I did not
repeat the mutation runs.
jmrplens added a commit that referenced this pull request Oct 22, 2026
Closes #1053.

This is the fourth layer of the authentication-budget stack, on top of
the transport-source bound of #982. It puts the three documents the
server publishes without a credential behind one token bucket per client
address, as the decisions of 2026-10-06 and 2026-10-17 on the issue set
out: on by default with figures I measured for it, `0` to turn it off,
one per-address row in the tenant policy register with the finding that
records why it has no process partner, one bucket shared by the three
documents, preflights counted, `/health` left out, and a fixed
`Retry-After` in the register.

## What was wrong

The SEP-2127 card (`/server-card`, `/mcp/server-card` and both under the
`--public-url` prefix), the legacy enumerating card
(`/.well-known/mcp/server-card.json`) and, in oauth mode, the RFC 9728
metadata are answered to anyone, before authentication, and nothing
bounded how often one caller could ask. The server-card extension asks a
server to rate-limit requests for its card. The legacy card is 141 KB on
the dynamic surface, so two addresses flooding it drew 141 MB/s of
egress from the process in my measurement, and every request is a full
trip through the HTTP stack.

## What changed

`cmd/server/discovery_rate.go` adds `discoveryRateLimiter`: a
`rate.Limiter` per client address in a table of at most 4096
(`tenancy.DiscoveryTrackedAddresses`), keyed exactly as the
authentication budgets key an address (`budgetKey(clientIP(...))`: the
peer, or behind `--trusted-proxies` the address `--trusted-proxy-header`
names, an IPv6 address by its `/64`). Its middleware charges every
request whose path is one of the documents' paths
(`discoveryDocumentPaths`, the same paths `serveHTTPOn` mounts the
documents at, prefix included), whatever the method, so a `304`
revalidation and a CORS preflight cost what a fetch does, and passes
everything else untouched. A request the bucket cannot pay for is
answered `429` with `Retry-After: 1`, the gate's JSON-RPC body with code
`-42900` and the words "Too many requests for the discovery documents
from this address. Retry later.", `Access-Control-Allow-Origin: *` and
`Retry-After` exposed, so a browser client can read the refusal the way
it reads the documents. The refusal is charged to no authentication
budget, and it is logged through the same once-a-window throttle the
other refusals use, in words that name the limit and not the address.

The middleware sits on the root chain, outside the CORS layer and inside
host validation. Outside CORS because `corsMiddleware` answers a trusted
origin's preflight itself before the mux is reached, so a bucket wrapped
around the document handlers would never see the preflights the decision
says to count. Inside host validation so that a request refused for its
`Host` reaches no document and costs nothing. A test in `cmd/server`
pins each side, and the e2e pins the first on the binary, where the
`--public-url` origin is trusted with no `--trusted-origins`. A table at
its bound gives a new address the place of the one used least recently,
and the five-minute sweep the authentication tables run on (`AUB-005`'s
interval, whose row now names this table too) forgets every bucket that
has refilled.

The settings are `--discovery-rate-limit-rps` and
`--discovery-rate-limit-burst`, and
`GITLAB_MCP_DISCOVERY_RATE_LIMIT_RPS` and
`GITLAB_MCP_DISCOVERY_RATE_LIMIT_BURST` through the HTTP env overlay,
both added to `prefixedNames` (and given no old spelling, below).
Defaults are 20 a second and 200, maxima 1000 and 10000, `0` for either
turns the bucket off, and a malformed, negative, NaN or out-of-range
value refuses startup with a message naming the flag
(`validateDiscoveryRateBounds`, called from
`validateHTTPPoolAndRateBounds`). Both are HTTP-only flags, so a stdio
run that is given one names it at startup and ignores it, and both are
announced in the HTTP startup record, zero included, so an operator who
turned the bucket off reads that it is off.

## The register and the specification

`RTC-008` is the new row: Allow, a Rate on `KeyAddress` with no stdio
key, class A, `Table: true`, `AtCapacity: EvictOldest`,
`ProtectsProcess` with no partner, configurable with `RefuseStartup` and
`ZeroOff`, decided by issue 1053 and carrying F-36. Its six values are
in `values.go` with their pins (`DiscoveryRate`, `DiscoveryBurst`,
`DiscoveryRateMax`, `DiscoveryBurstMax`, `DiscoveryTrackedAddresses`,
`DiscoveryRetryAfter`), the server and config constants are declared
aliases of them, and every enforcing site is declared on the row, which
`make check-tenancy` holds: 89 rows, 387 declared sites, 0 findings.

F-36 records the decision of 2026-10-17 that no bucket keyed on the
process stands beside this one, a departure from `INV-004`: a bucket the
whole process shared would let a flood from many addresses deny
discovery to every caller, a hosted client's shared egress address
first. Recording it needed one change to `Validate`: its `INV-004` check
looked only at requester keys, so a per-address row protecting the
process with no partner passed without a finding. It now holds a key
before admission to `INV-004` as it holds a requester's, since an
address is as mintable as a credential, and `RTC-008` passes by F-36. No
other row is affected; the tenancy suite says so.

`Retry-After` needed a source of its own, `RetryAfterRefill`, a register
constant of a bucket's own (`DiscoveryRetryAfter`), because the decision
asked for a fixed second rather than one derived from the bucket's
state, and `cmd/audit_tenancy`'s declarations name `discoveryRetryAfter`
as the read that satisfies it.

Two things differ from the plan this layer was written against, and in
both the issue and the code won. The plan said `StopCounting` at
capacity; I chose `EvictOldest`, read for a bucket as least recently
used, since a bucket has no expiry (the enum's comment now says so).
Neither rule limits a caller holding more addresses than the table,
which any IPv6 `/48` does: under eviction it cycles them and finds each
forgotten when it returns, and against a table that stopped counting it
would fill the table and then ask from one more address uncounted until
the sweep. Refusing newcomers at the bound would close the gap only by
denying discovery to every new address while the flood lasts, which is
what the missing partner avoids, so the gap is part of what F-36
accepts, and the row, the specification and the HTTP server page say so.
What eviction changes is the cost: an address that keeps asking stays
counted, so a flood has to keep cycling through more than 4096
addresses, drawing about one burst from each before it is forgotten,
where a table that stopped counting would let a few thousand single
requests from distinct `/64`s leave every newcomer uncounted until the
sweep. Evicting also forgets a record against its own address and takes
no holding from anybody (`INV-005`). The plan said to wrap the document
handlers; that would miss the preflights CORS answers first, as above.

## The old spellings

Adding the two variables to `prefixedNames` made `RetiredEnvUses` look
for their bare suffixes and report a `DISCOVERY_RATE_LIMIT_RPS` in
somebody's shell as "no longer read (removed in 3.1.0)", a name no
version ever read. The same was already true of the four authentication
budgets, which became settings in 3.1.0 after that release had stopped
reading old spellings. The last commit makes the settings with an old
spelling a closed list, `hadOldSpelling`, the 28 that 3.0.0 read, held
by a test to that count and to the settings the server still defines, so
a setting added later needs nothing for its bare suffix to go
unreported. The environment reference now says 28 settings and 22 bare
names, in English and Spanish, and names the six that have none. This
changes behavior for the four authentication budgets too: a bare
`AUTH_FAILURE_LIMIT` no longer produces the warning, which was wrong for
them.

Validation checklist for `RTC-008`:

- VAL-001: Allow.
- VAL-002: `KeyAddress`, the authentication budgets' key, with its mint
cost as the register already states it; no new key.
- VAL-003: class A, stated and reason unit the process.
- VAL-004: a ceiling on a rate, not a share.
- VAL-005: it protects the process and has no process partner, by issue
1053's decision, recorded as F-36.
- VAL-006: `EvictOldest` over a table of 4096, least recently used, for
the reason above, with the gap past 4096 addresses recorded under F-36.
- VAL-007: the gate's `429`, code `-42900`, `Retry-After: 1` from the
register, the words above, readable from any origin; on stdio the
documents are not served and the row has no key.
- VAL-008: charges nothing; no authentication budget sees a discovery
refusal.
- VAL-009: 20 a second and 200, maxima 1000 and 10000, configurable, `0`
turns it off, a malformed value refuses startup.
- VAL-010: no new telemetry. The middleware sits inside
`mcpotel.ServerMiddleware`, so a refusal reaches the HTTP server span
and metric as a 429 with its method, like every other gate refusal (that
layer records no route or path, by design), and the log line carries the
limit's two figures and nothing else, never the address, which a test
holds to that exact set of attributes.
- VAL-011: nothing on stdio, where no document is served; the two flags
are named and ignored there.
- VAL-012: the measurement below.

## The measurement (VAL-012)

`cmd/bench_resources` gains a `discovery` bound for `make bench-fairness
BOUND=discovery`: document verbs (the card, its revalidation with a tag
read before the arm, the legacy card, the metadata), each a row of its
own; a flood from one address per noisy lane; and a quiet population
that is a hosted client's egress address, eight lanes sharing one
address, each a user fetching the metadata and the card and then calling
`tools/call`. The bucket is per address, so the plan's arithmetic counts
what one bucket is offered rather than what one lane offers.
`BenchmarkDiscoveryDocuments` in `cmd/server` prices one request through
the real handler chain. I ran both on a 16-thread i5-14400 in a
container held to 5 cores:

- One request is a round trip of 40 to 95 microseconds whatever the
document, dominated by HTTP itself; a refusal costs about what the
metadata does. The SEP-2127 card is 1,371 bytes, the legacy card
140,934, the metadata 288, a refusal 132.
- Sixteen addresses flooding at 100 a second each: 0.25 and 0.30 cores
busy without the bucket, 0.26 and 0.22 with it; every flooding address
refused four requests in five with it in force, each document row served
its share; the shared egress address refused nothing in either arm; the
quiet tenant's `tools/call` at 10 to 15 ms at the median in both arms.
- Two addresses at 2,000 a second each: 0.42 to 0.47 cores without the
bucket, 0.37 with it, most of what remains being the refusals themselves
(about 92 microseconds each against about 110 served); the legacy card's
egress falls from 141 MB/s to about 1.6 MB/s; the shared egress address
refused nothing.
- The verdict of every run is `indistinguishable`, for the harness's
saturation reason: discovery requests at the rates one address can be
held to do not contend for the machine, so what the bucket bounds is
each address's draw, not the quiet tenant's latency.

That sized the defaults. At 20 a second one address costs the process at
most about 2.2 ms of processor a second and 2.8 MB/s of egress on the
legacy card. A hosted client's shared egress address has 72,000
discovery requests an hour, so a client that fetches the metadata and
the card on every connection connects 36,000 users an hour from one
address before its bucket runs dry. A burst of 200 lets a reconnect wave
of 100 users from one address through at once. `Retry-After` is a fixed
second, inside which a bucket refilling at 20 a second, or at any rate
of one a second or more, has a request to give again; at a configured
rate below one a second it is shorter than the refill, which the HTTP
server page says. 4096 addresses is the authentication tables' figure, a
few hundred kilobytes.

The first run used four times the rate per address, and it showed an
artifact of the open-loop schedule rather than of the bucket: a lane
cycles its four verbs on fixed ticks, a token refilled every fourth
tick, and two of the four documents got every token. The bound now
defaults five times the rate, and a test holds the ticks per token and
the verb count to sharing no factor.

## Documentation

The site's HTTP server page gains a section on the bucket in English and
Spanish, and the CLI, environment, configuration, remote deployment and
security pages name the two settings and what the address key means
behind a proxy: behind a proxy the server is not told to believe, every
caller shares the proxy's one bucket, so either name the real address or
turn it off. The HTTP server page also says what the fixed `Retry-After`
means below one request a second, and that a caller cycling more
addresses than the table holds is held back by no bucket, so a
deployment exposed to that limits these paths at its proxy as well.
`docs/development/configuration.md` lists both settings and the closed
list of old spellings, the specification records `RTC-008` and F-36,
`docs/development/cmd-utilities.md` and the `bench-fairness` comment
describe the new bound, and `CLAUDE.md` gains only the variables' row.

## How it was tested

The tests came first, and every new one is red on the parent as it stood
before the stack was rebased onto `main` (`d14bb760c`, now `5db3a1700`),
with this layer's test files copied onto its tree: `cmd/server`,
`internal/config`, `internal/tenancy` and `cmd/bench_resources` do not
build (`discoveryRateLimiter`, `DiscoveryRateLimitRPS`, `RTC-008`'s
constants and `documentRequest` are undefined); `cmd/audit_tenancy`
fails `TestRunMain_TheTree_PassesTheGate` on the gate's summary; the
four `TestDiscoveryRate_*` cases of `test/e2e/http` fail against the
parent binary (it refuses the two flags at startup, announces no bucket
and serves every fetch), and the stdio case fails on `flag provided but
not defined: -discovery-rate-limit-rps`.

- `cmd/server/discovery_rate_test.go`: the burst and its refill,
addresses kept apart, the least recent address evicted at the table's
bound, the sweep forgetting refilled buckets, zero turning the bucket
off, the document paths per mode and prefix, only the document paths
charged (a fetch, a preflight and a HEAD from one bucket; `/health`,
`/mcp` and unserved paths untouched), the refusal's status, body and
headers, its log line held to the limit's two figures and no other
attribute, the key through a trusted proxy and by `/64`, and
`serveHTTPOn` wired end to end in both auth modes under a prefix (200,
then 304, then a refused preflight from the trusted public origin, with
`/health` and `/mcp` still answered), a request with an undeclared
`Host` at every document path refused 403 and charged nothing, plus the
bounds validation table. The placement tests were shown red on mutants
of `serveHTTPOn`: with the bucket inside the CORS layer every preflight
is answered 204, in the unit test and in the e2e alike, and with it
outside host validation the 403s spend the bucket and the next good
fetch is refused. A log line given the address fails the attribute test.
- `internal/config`: `hadOldSpelling` held to the 28 settings 3.0.0 read
and to the settings the server defines, and `RetiredEnvUses` reporting
nothing for the bare suffixes of the four authentication budgets and of
the two discovery settings, red on the old loop over every setting.
- `cmd/server` flags, help, startup record and env overlay,
`internal/tenancy` (the row, its pins, F-36, the widened `INV-004`
check) and `cmd/audit_tenancy`.
- `test/e2e/http/discovery_rate_test.go` drives the real binary: each
document at every path under a prefix in both modes, the forwarded
address believed only from a trusted proxy, `0` through the flag and the
variable, and the shipped default refusing one address past its burst of
200. `test/e2e/stdio` names the two flags as ignored on stdio.
- `make test-e2e-http`: 703 tests, 8 skipped (the ones that need
Docker), none failed. `make test-e2e-stdio`: 246 tests, none failed (235
before the rebase below; `main` added eleven, #1469).
- `make check-tenancy`, `check-dead-consts`, `check-test-goroutines`,
`check-test-subtests`, `check-test-file-names`, `check-doc-tool-names`,
`check-action-ids`, `check-e2e-static`, `check-gateway-chars`,
`check-sdk-context` and `check-md-escaping` pass, the `e2e` suite
compiles, and `golangci-lint` with the `e2e`, `httpe2e` and `stdioe2e`
tags reports nothing on the touched packages.
- Statement coverage is 100% and condition coverage (gobco) complete on
every touched package: `internal/tenancy` 454/454, `internal/config`
436/436, `cmd/audit_tenancy` 994/994 (1016/1016 before the rebase below,
the 22 fewer being in the scaffolding #1479 moved out of the package),
`cmd/server` 2356/2356, `cmd/bench_resources` 1592/1592.
- `make coverage-mutants` on the files this changes, the others excluded
with `GREMLINS_FLAGS=--exclude-files`, 0 lived and 0 timed out
everywhere: `cmd/server` (`discovery_rate.go`, `main.go`,
`env_overlay.go`, `http_only_flags.go`) 434 killed; `internal/config`
(`config.go`, `http_overlay.go`, `env_name.go`) 176; `internal/tenancy`
(`decision.go`, `validate.go`, `decisions.go`, `decisions_allow.go`,
`decisions_admit.go`, `values.go`) 185; `cmd/audit_tenancy`
(`declarations.go`) 22; `cmd/bench_resources` (`fairness.go`,
`fairness_run.go`, `fairness_result.go`, `presenter.go`, `rpc.go`) 465.
The not-covered mutants (16, 9, 41, 5 and 9) are all in package-level
constant and variable initializers, which carry no statement counter,
the tool artifact `docs/development/testing/README.md` describes. Two of
them are lines this layer adds, `DiscoveryRetryAfter = 1 * time.Second`
and the discovery bound's defaults, and the tests pin both values; every
other one is on a line this layer does not touch.
- The site builds and lints, `i18n:check`, `facts:check`, `chips:check`
and `llms:check` pass, the Markdown tables are formatted, markdownlint
is clean and `scripts/check-doc-links.mjs` finds every link and anchor.

I then rebased the stack onto `main` at `a0e4e6cf6`, with no conflict
and every commit's patch unchanged, so the results above were measured
on the branch as it stood on `19746966465`, except the two figures that
say otherwise. Of the packages this layer touches, `main` moved
`cmd/server` and `cmd/bench_resources` in their tests and by one blank
line in `fairness.go` (#1472) and `cmd/audit_tenancy` in its scaffolding
(#1479). At this head `go build ./...`, `go vet`, `golangci-lint
--build-tags e2e` and `go test` on the five packages pass, each still at
100.0% of statements; `make check-tenancy` still reports 89 rows, 387
declared sites and 0 findings; `make test-e2e-http` and `make
test-e2e-stdio` pass; gobco on `cmd/audit_tenancy` is the 994/994 above;
and `make check-closed-servers` and `make check-godocs`, the two gates
`main` gained since (#1474, #1478), pass. I did not repeat the mutation
runs.
jmrplens added a commit that referenced this pull request Oct 22, 2026
Closes #1102.

This is the fifth layer of the authentication-budget stack, on top of
the discovery documents' bucket of #1053. It changes the words of the
gate's session refusal and nothing about when it refuses, as the
decision of 2026-10-17 on the issue set out: one answer for both cases,
no record of each credential's ended session IDs, an INFO line of its
own for a session the process does not hold, and the WARN line for
another credential's session kept.

## What was wrong

On `--stateless=false` the gate answers `404` to a request whose
`Mcp-Session-Id` names no session open for the credential it presents,
and the answer said "This session does not belong to the presented
credential." That is true of a session another credential opened. It is
false of the case a client meets every day, its own session that idled
out past `--session-timeout`, was deleted, went with its credential's
pool entry, or was opened before a restart or on another replica:
nothing that client presented belongs to anybody else, and the message
told it otherwise. Row ADM-007 of the tenant policy register described
only the first case too.

## What changed

`sessionOwnershipFailure` in `cmd/server/auth_gate.go` now answers "This
session is not open for the presented credential. Start a new session by
sending initialize without a session ID.", which is true of both cases.
The status, the code (`-32600`) and when it is answered are unchanged. I
kept one answer rather than two because answering the cases apart would
tell a caller whether another credential holds the ID, which INV-019
forbids, and because it would need a record of each credential's ended
session IDs, state keyed on a value the caller mints, with a row and a
capacity of its own.

`checkSessionOwnership` tells the cases apart in the log instead.
Another credential's session keeps its WARN line, `request rejected:
session ID does not belong to the presented credential`, word for word,
since it is true of that case and an operator may already alert on it.
An ID the process does not hold gets an INFO line, `request rejected:
session ID names no session open in this process`, through the same
once-a-minute throttle as the gates' other refusal lines, since a caller
can send any ID it likes. Neither names the ID.

Row ADM-007 in `internal/tenancy/decisions_admit.go` moves with the
gate: its resource is now "a session ID that names no session open for
the presented credential", its refusal prefix is the new first sentence,
and it records issue 1102's decision. It stays a rule with no value, no
table and no capacity, which is the half of the decision a test holds it
to.

The e2e phrase the go-sdk client pins expect
(`test/e2e/http/session_not_found_test.go`) and row 68 of
`docs/development/upstream-bugs.md` quote the new words. Row 68 no
longer says the call path shows the wrong cause; what a go-sdk v1.8.0
client is left with there is a true refusal on a connection that looks
alive, with an error that does not wrap `ErrSessionMissing`. The HTTP
server page, in English and Spanish, says the `404` answers both cases
alike, why, and which log line names each, and its refusal table now
names that row's condition as ADM-007 does, "an `Mcp-Session-Id` that
names no session open for the presented credential", where it said one
"this deployment did not open", which is false of the client's own
session that ended.

A last commit puts `resolve`'s doc comment back on `resolve`. It had
drifted above the `Mcp-Session-Id` header constant beside the check this
changes, so godoc read it as part of the constant's comment and
`resolve` had none.

## How I tested it

Tests first, each shown red on the parent as it stood before the stack
was rebased onto `main` (`3df394937`, now `b37e82899`) and green on this
branch:

- `TestDecisions_EachRowIsWhatTheSpecificationSays/ADM-007` pins the new
prefix, and `TestDecisions_SessionRefusal_IsOneAnswerByIssue1102` pins
the resource, the recorded decision, the absence of any value, table or
capacity, and the one refusal.
-
`TestMcpServerGate_CheckSessionOwnership_RefusesASessionFromAnotherCredential`
now holds every refusal (another credential's session, an ID never
recorded, a session whose credential was evicted) to the same code and
the same full text, and asserts the log: the WARN line for the first,
the INFO line for the other two, and neither for a request the check
lets through. On the parent the text differed and the INFO line was
missing.
- `TestMCPServerGate_EachRefusal_CarriesItsOwnStatusAndCode` drives a
real foreign session (its old case was in fact an unrecorded ID) and an
unrecorded one through the middleware. Since both are answered alike,
each also asserts the line the session check logged, WARN for the
foreign session and INFO for the unrecorded one, and every other case
that it logged neither; without that, a foreign session whose ID was
never recorded passes as one, which is how the old case went unnoticed.
I checked that both ways: with the foreign case's ID swapped for an
unrecorded one, the table without the log assertion still passes and
this one fails on the log line.
`TestMcpServerGate_Middleware_RefusesASessionMintedForAnotherCredential`
holds POST, GET and DELETE to the full text.
- `TestGate_SessionRefusal_IsOneAnswerOnTheWireAndTwoLinesInTheLog` in
`test/e2e/http` drives the binary: another credential's session and an
ID the process does not hold are answered with byte-identical bodies,
and the server's stderr carries the WARN line for the first and the INFO
line for the second. The two go-sdk pins of row 68 pass with the new
phrase.

Results on this branch: `go test ./internal/tenancy/... ./cmd/server/...
./cmd/audit_tenancy/...` passes; `make check-tenancy` reports 0 findings
over 89 rows; `make test-e2e-http` passes, 708 tests with 8 skipped for
want of Docker; statement coverage is 100.0% for `cmd/server` and
`internal/tenancy`; gobco conditions are 2356/2356 for `cmd/server` and
454/454 for `internal/tenancy`; `make coverage-mutants` over the whole
of each package, no file excluded, reports Lived 0 and Timed out 0 on
`cmd/server` (1301 killed, the five in `checkSessionOwnership` among
them, in 3 hours 7 minutes) and on `internal/tenancy` (255 killed). The
47 and 66 it calls not covered are the tool artifact
`docs/development/testing/README.md` describes, mutations in
package-level constant and variable initializers, which carry no
statement counter (the gate's error codes and message concatenations,
the HTTP timeouts and other durations, the env flag names, the listen
endings' and subscription errors' texts, the register's codes, values
and `iota` enumerations), plus two in `socket_mode_windows.go` and
`conn_refused_windows.go`, which are not compiled on Linux; none is on a
line this branch touches; `golangci-lint` is clean on both packages
under `e2e` and on `test/e2e/http` under `httpe2e`; `go test -tags e2e
-c` compiles `test/e2e/gitlab/...`; `make check-test-subtests`,
`check-test-goroutines`, `check-test-file-names`, `check-action-ids`,
`check-dead-consts`, `check-doc-tool-names`, `check-md-escaping` and
`check-e2e-static` and the godoc audit pass; markdownlint, the Markdown
table check, the doc link check, and the site's i18n, facts, chips,
llms, build and lint checks pass.

I then rebased the stack onto `main` at `a0e4e6cf6`, with no conflict
and every commit's patch unchanged, so the results above were measured
on the branch as it stood on `19746966465`. Of the packages this layer
touches, `main` moved `cmd/server` in two test files (#1472). At this
head `go build ./...`, `go vet`, `golangci-lint --build-tags e2e` and
`go test` on `cmd/server` and `internal/tenancy` pass, each still at
100.0% of statements, and so do the tests of `cmd/audit_tenancy`, also
at 100.0%; `make check-tenancy` still reports 0 findings over 89 rows;
`make test-e2e-http` passes with the same 708 tests; and `make
check-closed-servers` and `make check-godocs`, the two gates `main`
gained since (#1474, #1478), pass. The gobco figures count the
conditions of code `main` did not change, so they stand; I did not
repeat the three-hour mutation run.

## Tenant policy checklist

This changes a refusal's text and a row's resource, not a limit. VAL-001
to VAL-006, VAL-008, VAL-009 and VAL-011 are unchanged on ADM-007
(Admit, keyed on the session, class C, a rule, nothing kept, not
charged, no value, none on stdio). VAL-007 is the change: the gate's
`404` with `-32600`, no `Retry-After` and no challenge, answer "start
over", now in words true of both cases. VAL-010: with telemetry on, the
new INFO line is exported as a log record, as the WARN line beside it
already is; it carries no attribute but the throttle's count of repeats
it held back, so it names no session ID and no credential, and no
metric, span or reason changes. VAL-012 does not apply, since nothing
here protects one population from another.

The generated testing reference gains the new tests' entries when the
top of the stack regenerates it, as the stack's other layers leave
theirs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

e2e The end-to-end suite and its Docker fixtures security Security-related issue tooling The audit and generator commands under cmd/, and the Makefile targets that run them transport stdio and HTTP transports, the server process, and the transport e2e modules v3.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant