Repository navigation
Give tests a dead endpoint that keeps its port, and move the sixteen closed-server sites onto it - #1472
Conversation
…p their port A test that needs a server that is gone has been starting an httptest server and closing it, keeping the URL. Closing it frees the port, and whatever is given that port next can answer the request the test exists to see fail. DeadEndpoint holds a loopback port for the whole test and closes every connection it accepts before reading a byte. An accept that fails for any reason but the listener being closed, such as a process out of file descriptors, is retried after a pause, since a loop that ended there would leave the port held with nothing accepting and every request waiting out its deadline. StoppableServer answers through its handler until Stop, then hangs up on every request, the ones on connections kept from before included, and keeps its port. Each gives its port back in the cleanup it registers, which runs after every cleanup registered later and before any registered earlier, as their doc comments say, so a cleanup registered before either is asked for must not use its address. The package sits beside internal/testutil rather than inside it because internal/gitlab's own tests need it and testutil imports internal/gitlab. It is added to the test support the server binary must never link. Refs #1132
…ed port TestRunProbe_ATargetNothingListensOn_IsReportedRefused and TestRunProbe_Discovery took their "nothing answers" address from an httptest server they closed. The first ran alone, so only another process, such as a test binary of another package in the same coverage run, could be given its freed port; the second runs in parallel, so a parallel test of this package starting a server could be given it as well. Either could then answer the probe. The first was a test of its own, outside TestRunProbe_ExplicitTarget's parallel table, because a sibling starting a process could make its connect end in a reset that broke its "refused" assertion. The case moves back into that table on a dead endpoint and stops asserting "refused": a connection that is accepted and then closed ends with an EOF or a reset. It now holds what the probe is for, an unhealthy exit naming the target that gave no answer, and the separate test is gone. Discovery's peer that answers nothing is a dead endpoint as well. Refs #1132
Six verifier tests closed an httptest server to get an instance that does not answer, all six of them in parallel, so another test of the package could be given the port in between. Five now use a dead endpoint, and their comments no longer describe a refused connection. TestNewGitLabVerifier_NetworkErrorWithCache needs its instance to answer once and then stop, and its verifier and cache are bound to the URL, so it uses a stoppable server. Its comment now says what it checks: the test deletes the cached entry itself, so what it holds is that a failed re-verification admits nothing and caches nothing. Refs #1132
…e start TestInitialize_ServerDown and TestDetectToken_NoAnswer_IsNothingKnown closed a server to get an unreachable URL, the first one from stubVersionServer. The two credential probe tables built their client on a live server and closed it for the unanswered row. Building a client sends nothing, so those rows now build it on a dead endpoint, through credentialEndpoint, and the other two take the dead endpoint directly. Refs #1132
…ble row TestRevalidateAll_EvictsOnlyOnCredentialVerdict closed its server for the unreachable instance, leaving a port another listener could answer on with a status the row does not expect. Refs #1132
…ests Three sequential tests closed an httptest server to get an instance that cannot be reached. They now take a dead endpoint, and the apidocs test no longer calls the failure a refused connection. Refs #1132
…unMain main was only ever run in a child process the tests start, and the coverage counters of a child are not the test's, so the package stood at 97.0% of statements with main at none. main now hands its arguments, the environment and both streams to runMain, which parses a flag set of its own and returns the status, and ends the process through exitProcess. The child-process tests stay: they hold the streams and the status as make and CI read them. Two tests in the test process cover runMain's three ways out (a flag nobody defined, a request for help, a run) and main's hand-off of the status. The usage line now names the command rather than the path of the binary go run built. Refs #1132
The provider package stood at 96.4% of statements. Most of what no test reached was a failure path of one adapter that another adapter's tests already held: a body that is not JSON was held for Anthropic only and is now held for OpenAI and Google too, and the transport test runs over all three. The rest is covered where it lives: an answer whose body ends before its declared length, a request that cannot be built (a payload that does not encode, an endpoint that does not parse), an empty option in a spec, a result that is not JSON and a field an earlier step produced read through resolveTruth, the text block of an assistant turn that is not sent as a call, and the approval of a destructive action on the individual surface, which the test's own comment promised and did not check. The unknown-provider case of New was refused for its missing key before the provider was looked at, so it held nothing about the provider; it is now given a key and holds the refusal's wording. Four branches no input of a run can reach are now reachable or gone. buildFakeCatalog is handed the catalog builder and the fake encodes its calls through the variable encodeCall, so a test can make either fail, and a test replaces the cached catalog error to reach the path that reads it. producedValue decodes the result once and walks the decoded value, where it decoded every level again and then decoded the value it had just decoded a last time, and normalizeSchema returns what json.Marshal answers, since encoding what was just decoded cannot fail. Refs #1132
The loop in Fetcher.request counted its attempts in its own condition and also broke out after the last one, so the condition was never false and gobco reported it as evaluated one way only. The loop now has no condition and ends through that break, which is the one way out it ever took. The status the loop classifies was a switch with no tag, whose case expressions gremlins cannot see into, so six mutants of it were reported uncovered although each dies when applied by hand. It is a chain of ifs now, with retriedStatus naming the statuses worth asking again for, and gremlins measures and kills each of them; the testing README's account of those six says so. Refs #1132
… its mutants The provider package was at 100% of statements, but gobco found twenty conditions evaluated one way only, and gremlins left 28 mutants lived and 21 uncovered. Every one is now answered. What the tests did not hold, they hold now: the echo of each adapter is handed back only for an assistant turn and only when it is a non-empty list of blocks, a content whose parts decode, or a message naming its role; a rebuilt Anthropic call goes back as the call the model made; a block or a part the record has no kind for is passed over; a Google call the provider did not name is named for its position counted from one, and so is the fake's call for its step; an answer with no candidate names a block reason only when there is one; arguments of null are a malformed call; the wrong-args variant plants the first fact alone and the perfect one plants nothing; a meta call with no argument carries no params; an individual tool name is kept only for the action the call identifier resolves it to, tested through a catalog the test builds, since the real one has no name a sibling took; a produced value is refused for a step before the first; every side of the HTTP status classification, a detail of exactly the bound's length, the default client's timeout and the given client, the latency measured in milliseconds, a free input or output alone, a price read this instant or exactly at the window's edge, the age and the window in days, and a temperature of two, below zero, or with two decimals. Four switches with no tag are chains of ifs now (failureStatus, the price date check, googleBlocks and resolveTruth), because gremlins cannot see into the expressions of their cases and reported each of them as uncovered whatever the tests held. Two checks that could never decide anything are gone: a Google call's args are checked for length after json.Valid, which already refuses empty input, and an echoed content's parts were checked for length before a decoding that refuses them empty. The note of the Gemini price is one literal, since an operator in a package-level initializer carries no statement counter. What remains is two uncovered mutants in the constant 180 * 24 * time.Hour of maxPriceAge, for the same reason; the price tests hold the window to 180 days at its edge and past it. Refs #1132
go doc reads the comment of a const group, not of the specs in it, for the one exported name a group declares, so the godoc audit reported DefaultAreasURL as undocumented although its spec carries a long comment. The group now says what its constants are for. Refs #1132
The godoc audit reported three things in packages this layer changes, none of them new here. cmd/bench_resources had a second package comment, because fairness.go's file comment sat directly on its package clause; a blank line makes it the file comment it is meant to be, and the package comment stays in doc.go. internal/serverpool's StartIdleEviction had its comment two functions above it, run into recoverSweep's, so the method was undocumented and recoverSweep's comment opened with another function's text; it is back on the method. Under --include-tests, eleven tests of cmd/internal/apidocs, the one this layer moves onto a dead endpoint among them, and three of internal/serverpool had no doc comment; each now says what it holds. Refs #1132
There was a problem hiding this comment.
jmrplens has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 57 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The base URL was a string concatenation with a literal scheme, which SonarCloud reads as a request sent over plain HTTP. It is a loopback address no request is meant to reach, so it is built as a url.URL, the form the bench harness uses for its own loopback servers, and reads the same. The local variable that held it is renamed so it no longer shadows the url package. Refs #1132
There was a problem hiding this comment.
jmrplens has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
|
… distinct-credential budget's reason from it (#1480) Refs #954 The distinct-credential budget gave as its reason that GitLab rate-limits failed authentication per source address, so a token sprayer spends the deployment's standing with GitLab and the throttle GitLab eventually applies lands on everyone using the server. GitLab's documented failed-authentication ban covers Git and the container registry, not the API, and whether an invalid-token REST request counts against GitLab's per-IP limit on unauthenticated requests was an open question (OQ-11, finding F-25). This is the first layer of the issue: I measured the answer on GitLab 19.5 and rewrote the reason to it. The keying change of the budgets is the next layer. ## The probe I added `TestAuthThrottle_InvalidTokens_SpendTheAddressUnauthenticatedAllowance` to `test/e2e/gitlab/ce`. It reads the instance's unauthenticated API throttle, turns it on at four requests in two minutes, and waits the minute each GitLab process keeps its settings for. It then sends invalid-token `GET /api/v4/user` requests itself, in `PRIVATE-TOKEN` as the server's legacy verification sends a token, has the real binary in HTTP mode verify an invented token, and sends one more invented token as a Bearer token, as the OAuth verification sends one, reading what GitLab counted off the `RateLimit-*` headers GitLab 19.5 writes on every counted response. Once GitLab throttles the address, it sends an anonymous request and the run's own token, starts three servers in HTTP mode (one per surface) that verify the run's token and serve `user.current`, and checks the address is still throttled afterwards. It then waits out the window to see whether a ban follows, writes the settings back, and checks nothing counts an invalid token any more. GitLab 19.5 counts that throttle with Labkit rather than Rack::Attack. Every Labkit cohort enforces by default (`rate_limiter_use_labkit_rack_cohort_{1,2,3}_enforce`, `default_enabled: true` at v19.5.0-ee), Rack::Attack then safelists every request as `labkit_fully_enforced`, and every line of the run's `api_json.log` says so. Labkit's counter is a Redis key that the first request it counts creates with the period as its lifetime (`lib/labkit/rate_limit/evaluator.rb` at labkit-ruby v5.5.0), so a window opens with the first counted request and ends a period later, while the `RateLimit-Reset` and `Retry-After` GitLab writes describe a window aligned to the epoch (`RequestThrottleData.from_labkit_result`). The probe therefore starts from a request counted as the first of its window, and waits for a window to end a full period after an answer counted in it (the answer that opened it, for the window it spends), never by those headers. Its last verdict tells a ban (a 429 or a 403 a full period after the window opened) apart from a count it did not expect (a 401 counted as anything but the window's first request). `TestThrottleWindow_OpensWithItsFirstCountedRequest` and `TestNextWindowProblem_TellsABanFromAnUnexpectedCount` hold that model with no GitLab, on the three calibrations of the recorded run. The throttle is an instance-wide setting, and the issue decided that no complete run may turn it on: a run cut short before the probe's cleanup would leave it on for every package after it. So the probe runs only when `E2E_AUTH_THROTTLE_PROBE=true`, only in Docker mode, and on its own. `run-docker-e2e.sh` now refuses to start a complete run (`E2E_GATE_SKIPS=true`) with that switch set, read from the environment, the file `E2E_ENV_FILE` names or the repository `.env`, the same places it reads the external network switch from, through a new `complete_run_value` in `skip-gate.sh`. That lookup sources the file, and the harness reads the same file with godotenv, which also takes `KEY: value` and `KEY = value` and reads on past a reference to a variable nobody set, so a switch written that way reached the probe without the script seeing it. A complete run the script does not refuse therefore starts with `E2E_AUTH_THROTTLE_PROBE=false` exported, and the process environment outranks every file the harness reads. `scripts/e2e_skip_gate_test.py` covers the lookup, the refusal, the export and its place in the script. A complete run skips the probe, and the skip gate fails a complete run on any skip nothing declares. So I declared the skip in `cmd/audit_e2e_coverage/skip_declarations.go` under a new category, `opt-in-probe`: a probe that changes an instance-wide setting, which a complete run must never run. It is not a setting a run left out, which the table still refuses to excuse. `test/e2e/README.md` has a new section on the probe, and `cmd-utilities.md`, `testing.md`, the header of `run-docker-e2e.sh` and the test-file counts in `CLAUDE.md`, the repository map and the testing guide's tree follow the table, the refusal and the export (the testing guide's tree also had stale common and ee counts, now 175 and 48). ## What GitLab 19.5.0 answered I ran the probe once, alone, on `gitlab/gitlab-ce:19.5.0-ce.0` (revision 4c9e34e1faa) on the Docker stack, and it passed in 425 s. Every request from the test process and from the server children reached GitLab as the same address, according to `api_json.log` and nginx's access log. That run was of the probe's first version, before it modelled Labkit's window and before it sent a Bearer token, so its `window_left` column is the epoch-aligned `RateLimit-Reset` minus `Date`, which the probe no longer reads. The probe as committed here has not run against GitLab. It needs one run on its own before this merges, with the command in the README's new section and `GITLAB_IMAGE=gitlab/gitlab-ce:19.5.0-ce.0`, and that run's lines take the place of the record below. ```text throttle as found: {Enabled:false Requests:3600 Period:3600} off: invalid token 1..6: 401 (no throttle) server refuses an invented token 401 on: calibration 1: 401 throttle_unauthenticated_api observed=1 remaining="3" window_left=51s on: calibration 2: 401 throttle_unauthenticated_api observed=2 remaining="2" window_left=1m59s on: calibration 3: 401 throttle_unauthenticated_api observed=1 remaining="3" window_left=1m58s server, HTTP mode, verifies an invented token: refused 401 on: invalid token after the server's verification: 401 throttle_unauthenticated_api observed=3 on: invalid token 1 toward the limit: 401 throttle_unauthenticated_api observed=4 remaining="0" on: invalid token 2 toward the limit: 429 throttle_unauthenticated_api observed=5 retry_after=1m58s on: anonymous request past the limit: 429 throttle_unauthenticated_api observed=6 on: the run's token past the limit: 200 (no throttle) three fresh servers verify the run's token and serve user.current: PASS on dynamic, meta, individual on: invalid token after the servers were served: 429 throttle_unauthenticated_api observed=7 on: invalid token in the next window: 401 throttle_unauthenticated_api observed=1 restored: invalid token 1..6: 401 (no throttle) server refuses an invented token 401 ``` With the throttle off (as a self-managed instance ships it), nothing counts an invalid token. With it on, the answer to OQ-11 is yes: every request carrying an invalid token in `PRIVATE-TOKEN` is counted once against the address, including the server's own legacy verification of one. The source agrees: `Gitlab::Auth::RequestAuthenticator` rescues the invalid token's authentication error to nil, so the request is unauthenticated and the throttle keys on its address. Past the limit the address loses its whole unauthenticated allowance, so an anonymous request is answered 429 too. A token GitLab accepts is unaffected: it was answered 200 and counted by no unauthenticated limit (GitLab's authenticated API limit, which counts per user, was off, as it is by default), and freshly started servers verified and served it throughout. No ban followed: in the next window the invalid token was answered 401 and counted from 1 again. The first version's record has no Bearer token in it, and the OAuth verification sends one. GitLab 19.5 reads `PRIVATE-TOKEN` and a Bearer token through the same finder (`find_user_from_any_authentication_method`, `lib/gitlab/auth/auth_finders.rb` at v19.5.0-ee), whose refusal `RequestAuthenticator` rescues the same way, and GitLab.com, which runs its throttle on, counted both alike when I asked it on 2026-10-21 at 01:34 UTC: ```text GET https://gitlab.com/api/v4/user, one window, one address PRIVATE-TOKEN glpat-<invented> 401 throttle_unauthenticated_api limit=500 observed=2 Bearer <invented, no prefix> 401 throttle_unauthenticated_api limit=500 observed=3 Bearer glpat-<invented> 401 throttle_unauthenticated_api limit=500 observed=4 no credential 401 throttle_unauthenticated_api limit=500 observed=5 ``` So the comment, the security page and the specification say what the 19.5 run observed for `PRIVATE-TOKEN`, and what GitLab.com and the source say for a Bearer token, until the run above records it on 19.5 too. Every count in the record is the probe's own. Calibration 2's count of 2 is calibration 1, sent 52 s earlier into the same Labkit window: the first version of the probe read the epoch-aligned reset as the window's end, waited it out and sent calibration 2 into the window calibration 1 had opened. The same reading made the run pass by timing. Calibration 3 opened the window the probe spent, which ended at 17:08:02.122, and the wait for its end, the throttled 429's `Retry-After` plus a second counted from a later 429, sent the next-window request at 17:08:10.489, 8.4 s after it. Started from a calibration with between 60 s and about 110 s left by the epoch, the same arithmetic would have sent it inside the window and reported a ban that did not happen. That is what the window model above fixes. Its counts and verdicts are the first version's, and only the waits, the starting window and the one Bearer request moved; the Bearer request is the fourth counted one of the window, so the request past the limit is still the fifth. ## What lands on the server The scenario cannot show what the server does with a token GitLab throttles, because `Env.RefusedAdmission` fails a test whose credential is admitted, and changing the harness is outside this layer. So I checked it by hand on the same stack after the run. I turned the throttle on again, spent the window with direct invalid tokens (429 from the fifth on), started the staged binary in legacy HTTP mode, and posted `initialize` with an invented token. GitLab answered the verification 429, which is no verdict on the token, and `verifyCredential` admits a credential it got no verdict for, as it does when GitLab is unreachable. The gate answered `initialize` 200 and the pool logged `created new entry` with `scopes_detected: false`. The build took 16 s and five more counted requests, because the entry's token and tier lookups retried against 429s. In OAuth mode the same 429 is an upstream error and the gate answers 503 without charging anything (`TestOAuth_ThrottledUpstreamIsNotBlamedOnTheToken`). So the rewritten reason says what was observed: a sprayer spends the deployment's unauthenticated allowance with GitLab, not its users' standing. What lands on the server once that allowance is spent is that its verification of every token GitLab does not accept, every invented one among them, gets no verdict: in legacy mode an invented token then gets a pool entry, and in OAuth mode a 503. A token GitLab accepts is still verified and served. The comment on `DistinctTokenBudget` and the security page in English and Spanish say this and name 19.5, and the note in `internal/config` and the paragraph in the tenant policy specification beside F-25 say what GitLab answered. The comment no longer calls the distinct count the earliest signal of that spending, which it is not at the defaults: a sprayer at ten or more distinct tokens a minute meets the failure lockout at its tenth, long before the fiftieth. What the distinct count does is bound the sprayer the lockout cannot, one kept under ten a minute, and block for longer each time it is reached (a minute, ten, then an hour), where the lockout forgives after a minute. The security page and the comment also give GitLab.com's limits with their dates: in October 2026 it answered with 500 unauthenticated requests a minute per address, and GitLab announced 60 an hour for unauthenticated traffic from 19 October 2026, which its rate limits page still lists as proposed and not in effect. The budgets' defaults do not hold a sprayer below that 60, since the failure lockout alone lets ten through a minute. F-25 stays on `AUB-001` and `AUB-003` until the keying layer of this issue removes it. I think the legacy admission under a throttled GitLab deserves an issue of its own. While GitLab throttles the deployment's address, the legacy door is open to every invented token: each one builds a pool entry (at the pool's bound, evicting another credential's quiet entry, `POL-002`) and holds a probe slot (`POL-006`) through the retries. The two doors answering the same state of GitLab differently is a finding by the specification's own rule, carried by the rows it concerns, and `Validate` refuses a finding no issue carries, so the specification in this layer records only what GitLab answered and leaves the two doors' answers to that issue. Once it is filed, it becomes the next finding on `ADM-001` and `ADM-002`, in the specification's findings table and in the paragraph beside F-25, with the keying layer's register change or before it. This layer changes no server code. ## The package the skip declaration touches The declaration puts this layer in `cmd/audit_e2e_coverage`, so I measured the whole package. On the parent, exported in full (a `git archive` leaves out `test/` and `docs/`, which the package's own tests read), `make coverage-mutants PKG=./cmd/audit_e2e_coverage` reported Killed 671, Lived 3, Not covered 88, the same as this branch before the third commit. The three that lived were real gaps, both in code this branch does not otherwise touch: `report()` of the fine-grained fold behaved the same under `||` as under `&&`, since no test had sessions without calls or calls without a session line, and `enclosing`'s half-open span was never asked about at its edges, which no site the scanner reads can reach. The third commit asserts both directly, and the package now reports Killed 674, Lived 0, Not covered 88, on the final head too, with the same 88 positions. None of the 88 has a statement counter to be covered by: they sit in `switch { case ... }` expressions and package-level initializers, the tool artifact `docs/development/testing/README.md` describes, so I applied each one by hand to a copy of the tree and ran the package's tests against it. 53 die: 50 under `go test` as it runs, and three more whose `||` turned `&&` makes a condition that cannot hold, which go vet refuses and the suite fails with vet off as well. The other 35 turn a `+` between two string literals into a `-`, which does not compile: 29 in `portmap.go`, four in `runtime_record.go` and two in `static.go`. gobco reports 1318 of 1318 conditions. ## How I tested it - The probe's own unit tests, with no GitLab: `TestThrottleAnswer_ReadsTheRateLimitHeaders`, `TestThrottleWindow_OpensWithItsFirstCountedRequest` and `TestNextWindowProblem_TellsABanFromAnUnexpectedCount`. Against the probe as it stood before the window model they do not build (`answer.At undefined`, `undefined: opensWindow`, `undefined: windowMargin`, `undefined: nextWindowProblem`); on this branch all three pass with their twelve subtests. - The probe itself, once on GitLab 19.5.0 CE in its first version, as recorded above. The version committed here still needs its one run. - The refusal and the export: `scripts/e2e_skip_gate_test.py` covers the lookup, the refusal, a run of the lifecycle script with the switch in the environment and in the named file, and a complete run started with the switch written as `KEY: value`, as `KEY = value`, after a reference to an unset variable, and not at all, whose stand-in `docker` records the switch each call inherits. Against the parent's scripts the 33 tests end with 14 failures and 3 errors, those four cases among them, seeing the switch unset rather than `false`, and the placement check finding no export; on this branch all 33 pass, among them a run not asked to be complete, which keeps the caller's `true`. shellcheck is clean on both scripts. - The skip a complete run sees, recorded against the same stack without the switch and fed to `audit_e2e_coverage -check-skips -runtime ce`: on the parent, `1 skipped on ce, 0 declared, 1 undeclared` and exit 1; on this branch, `declared (opt-in-probe)`, `1 skipped on ce, 1 declared, 0 undeclared, 0 stale declarations` and exit 0. - `go test -tags e2e -c` of `./test/e2e/gitlab/...`, `go vet` and `golangci-lint --build-tags e2e` on the CE package, `cmd/audit_e2e_coverage`, `internal/serverpool` and `internal/config`, all clean. - Statement coverage of `cmd/audit_e2e_coverage`, `internal/serverpool` and `internal/config` is 100%. The last two change only in comments, and I measured them anyway: gobco 418 of 418 and 424 of 424; gremlins Killed 215, Lived 0, Not covered 6 on `internal/serverpool` and Killed 205, Lived 0, Not covered 9 on `internal/config`. Applied by hand, all nine of `internal/config`'s not covered die and four of `internal/serverpool`'s six, the other two being a `+` between string literals in `token.go`. - `make check-tenancy`: 87 rows, 0 findings. `make tenancy-code-identity`: against the plain parent, 7 sections differ besides the line table, the documented case of a change that inserts lines into the server's source, here comment lines in `distinct_token_budget.go` (30) and `config.go` (3). Against the parent with blank comment lines inserted at the same positions, whose non-comment lines are the parent's at this branch's line numbers, the two binaries are identical in every allocated section besides `.gopclntab`. - `make check-e2e-static`, `check-test-file-names`, `check-test-subtests`, `check-test-goroutines`, `check-dead-consts`, `check-md-tables` and `check-doc-tool-names` pass. markdownlint on the changed pages, the em dash check over the branch and `scripts/check-doc-links.mjs` are clean. The site's `i18n:check`, `facts:check`, `chips:check`, `llms:check`, `build` and `lint` pass. - `gen_testing_docs --check` reports the generated counts in `testing.md` behind by the test functions and files this branch adds. That is the committed artifact this stack regenerates once at its top. - I rebased the stack onto `main` at `a0e4e6cf6`, with no conflict and every commit's patch unchanged. The figures above were measured on the branch as it stood on `19746966465`. Of the packages they measure, only `internal/serverpool` moved on `main` since, in the place of a doc comment and in its tests (#1472); `main` also changed files under `test/e2e` and `docs/` that the tests of `cmd/audit_e2e_coverage` read, which is why I ran those tests again here. At this head `go build ./...`, `go vet`, `golangci-lint --build-tags e2e` and `go test` on `cmd/audit_e2e_coverage`, `internal/config` and `internal/serverpool` pass, each at 100.0% of statements, the e2e suite compiles, the skip gate's 33 Python tests pass, `make check-tenancy` reports 87 rows and 0 findings, and `make check-closed-servers` and `make check-godocs`, the two gates `main` gained since (#1474, #1478), pass.
…e /48 and name the unix-socket peer (#1481) Closes #954. This is the second layer of #954. The first corrected the distinct-credential budget's stated reason from what GitLab 19.5 answered; this one changes the key every authentication budget before admission is charged to, which is what the issue opened with, and adds the bound one level up that the decision of 2026-09-28 asked for, with the constants and the /64 the decision of 2026-10-17 fixed. ## What changed The budgets were keyed on the whole client address. An IPv6 caller holds at least a /64 and can use any address in it, so rotating the low bits handed it a fresh fast budget and a fresh distinct-credential budget on every request, and nothing short of the coarse transport-source budget (500 distinct keys a minute, and only behind a trusted proxy header) ever cut it off. Every caller on a unix socket, meanwhile, shared whatever `RemoteAddr` spelled there, and `--trusted-proxy-header` could not separate them, because a header was believed only from a peer that parses as an address. I added `budgetKey` in `cmd/server/client_ip.go`: a whole IPv4 address, an IPv4-mapped address as the IPv4 address it names, and an IPv6 address by the fixed /64 it lies in. Both doors read the fast budget's and the distinct-credential budget's key through it, and the transport source (`AUB-002`) is keyed the same way, so a proxy reaching the server over IPv6 is one source however it rotates its own low bits. A caller holding a /48 holds 65536 /64s, so I added the aggregate the decision asked for as register row `AUB-006`: the /64s of one IPv6 /48 that had a credential refused, counted on the distinct-token budget's machinery with the /64 as the distinct value (held as a digest, in a table `AUB-004` caps with the other two), in `AUB-003`'s window and up `AUB-003`'s ladder, at `tenancy.AuthNetworkFailingPrefixes = 10`, a register constant no flag sets. I read "failing /64s" as /64s that had a credential refused, the same thing the distinct-credential budget counts, so a /64 whose clients only forgot their header does not move it. Its windows fall back to their defaults when a zero switches `AUB-001` or `AUB-003` off, so turning those off does not turn it off (`INV-015`); an IPv4 address and a unix-socket peer have no /48 and are not counted. What the fixed prefixes cost is stated rather than assumed away. Callers that share a /64 share its budgets, and on IPv6 that is more than the IPv4 NAT comparison brings to mind: the hosts of one IPv6 subnet, which on private IPv4 each had a key of their own, the customers a provider numbers from one shared /64, and every link-local caller, since the zone is dropped and fe80::/64 is the same prefix on every interface. A stateful NAT64 gateway changes nothing, since the server sees the gateway's IPv4 address and keys it whole, but a stateless translator in front of an IPv6-only server (SIIT-DC, RFC 7755) writes every IPv4 client into its RFC 6052 prefix, so under `64:ff9b::/96`, or any prefix of /64 or longer, they all share one /64, and under any prefix of /48 or longer one /48, where each had a key of its own before. The aggregate counts a refused credential whoever presents it, so an organization whose whole network is one /48 reaches its ten when callers on ten of its subnets present an expired or revoked token inside the distinct-credential window, and the /48 is then refused for a minute, then ten, then an hour, for every credential the deployment does not already serve; in legacy mode that is every token outside the pool, which `--max-http-clients` bounds at 100 by default. A /48 is not always one organization's either: a provider that delegates each subscriber a /56 or a /60 puts 256 or 4096 of them in one, and a mobile network gives each device a /64 of its own (RFC 6459), so ten unrelated subscribers presenting an expired token inside the window block the /48 for every other one, and no flag turns the aggregate off. The security page says all of this. The ten is the decision's figure and a register constant, so I left it as it is. A connection accepted on a unix socket is now the one key `unix`, read off the listener's address on the request context rather than off `RemoteAddr`, which is `@` on Linux, empty or a path elsewhere. `unix` is accepted in `--trusted-proxies` and trusts the socket's peer and no address, so the proxy on a socket names each caller exactly as a proxy on loopback TCP does. The host guard asks the same question through the same set (`trustedProxies.trustsPeer`), so the two cannot read the list differently. Telemetry counts the aggregate's refusals under a fourth reason, `network_aggregate`, since `authBlockCounters` drops a reason it does not know. `longestAuthBlock` now takes the four budgets as a list in the order that breaks a tie. ## Release note: a breaking startup check, and budgets shared by /64 Two pairings of the proxy flags with `--http-addr` now refuse startup, because each trusts nobody while reading as if it trusted someone. A unix-socket `--http-addr` whose `--trusted-proxies` names addresses and no `unix` used to start with `--trusted-proxy-header` silently ignored, so every caller on the socket shared one budget; it now stops with `--trusted-proxies does not name unix and --http-addr ...`, and the fix is to add `unix` to the list. `unix` in `--trusted-proxies` on a TCP `--http-addr` is refused as well. A socket listener with neither proxy flag starts as before. The budgets also share differently. Every address of one IPv6 /64 now spends one fast budget and one distinct-credential budget, so the hosts of one IPv6 subnet share them as callers behind one IPv4 NAT share an address's, the IPv4 clients a stateless translator (SIIT-DC) writes into `64:ff9b::/96` all share one /64, and ten /64s of one /48 that had a credential refused inside `--auth-distinct-token-window` block the whole /48 for every credential the deployment does not already serve. A credential the deployment already serves is still exempt from every block. ## The register and the documentation `IDN-004` and `IDN-005` derive at `budgetKey`, `AUB-001` to `AUB-003` record issue 954 and carry F-14 and F-25 no longer (F-34 stays for #958), the key vocabulary gains `network` with its mint cost and evidence, the address and source evidence state the /64 and the unix key, `AUB-004` names the /48 table among those it caps, and `AUB-006` is the eighty-eighth row with its frozen pin and its entry in `specRequirementIDs`. The specification's key table and findings say the same, with what the /64 and the /48 cost. I did not declare the two startup refusals as register refusals, as none of `validateTrustedProxyConfig`'s existing refusals is declared either. The site pages change in English and Spanish together: the security page's budget table gains the aggregate, with its fixed ten and its configurable window, its introduction names the transport-source budget a trusted proxy header adds, and a paragraph says what an address is, who shares one (a stateless translator's IPv4 clients among them) and what the /48 costs, to an organization whose network is one and to the unrelated subscribers a provider places in one; the TLS page's unix-socket section, which told a shared deployment to give the socket up, now shows `--trusted-proxies=unix` and the two refused pairings, and the remote deployment page's socket unit and its table of shapes carry it with the header it requires; the HTTP server, CLI, configuration and environment references name `unix` and the /64 for both address budgets, and the CLI reference says the /48 aggregate counts in the distinct-credential window; wherever a page said the header is believed only from a listed address it now adds the socket's peer when `unix` is listed; the troubleshooting rows for a 429 name the /48 and `unix`; the enterprise overview's card for the TLS page describes what that page now says; the telemetry page names `network_aggregate`. `docs/development/configuration.md`, `http-invariants.md` and the copilot instructions follow; the flag usage, the curated help and its ENVIRONMENT block name the /64 for both address budgets, and so do CLAUDE.md's two rows for them. The refusal of a `--trusted-proxy-header` with no list names `unix` beside addresses in its remedy, since on a socket listener a list of addresses alone is the pairing the next startup refuses. ## Validation checklist `AUB-006` is the new row; `AUB-001` to `AUB-003` keep every answer below but the reading of their key, which VAL-002 covers. - VAL-001, question: admit, like every budget before admission. - VAL-002, key: `network`, new in the vocabulary (`internal/tenancy/key.go` and the specification's key table): the IPv6 /48 a charged client address lies in, read by `networkKey`, minted as an address is, with its evidence that another value is another assignment while a provider may give one end site many /64s inside one /48, and may put many unrelated subscribers inside one. `AUB-001` and `AUB-003` stay on `address` and `AUB-002` on `source`, whose evidence now states the /64 and the `unix` key; neither mint cost changes. - VAL-003, class: A, before admission where no tenant exists yet, like the other budgets. - VAL-004, ceiling or share: a budget, a ceiling on what one /48 may spend that promises nobody a share. - VAL-005, process partner: it protects no process resource; its table is one of the three `AUB-004` caps. - VAL-006, at capacity: the /48 table is a `DistinctTokenBudget` capped at `AuthTrackedSources` (4096) keys, and like the distinct-credential table it stops counting a new key once full rather than evicting a record, since the records there are the evidence (`AUB-004`'s second refusal). - VAL-007, refusal: the gate's, at both doors: `429`, `-42900`, `Retry-After` the longest block holding the request and never under a second, no challenge, retry-later class, in every budget's words (`Too many failed authentication attempts from this address. Retry later with a valid token.`); a credential the deployment already serves is exempt, as from every budget. On stdio: none. - VAL-008, charges: a credential GitLab refused, at both doors and from the OAuth door's memory of a refusal too, and nothing else: not a request that carried no credential, and none of the failures the register declares uncharged. `TestDecisions_NetworkAggregate_IsTheConstantIssue954Added` holds that failure by failure against the register's `Failures`, and `make check-tenancy` (G7) holds the code to it. - VAL-009, value: `tenancy.AuthNetworkFailingPrefixes = 10`, `Source: Constant`, pinned in `TestValues_HoldTheirPins`; no flag or variable, so no bounds of its own and no malformed-value policy; zero is not applicable. Its window is `AUB-003`'s and its ladder's step `AUB-001`'s, each its default when the operator's is zero. - VAL-010, telemetry: `gitlab_mcp.auth.blocks` counts its refusals under the new reason `network_aggregate`, the reason being the metric's only dimension and the address not one (`INV-013`). The refusal's log line names the budget beside `ip` and `source`, as it already did for the other three, throttled like every refusal a caller can cause. - VAL-011, stdio: not applied (`StdioKey` none); stdio has no door. - VAL-012, measurement: not applicable. The aggregate does not claim to protect one population from another; it bounds what one /48 may spend, and its cost to an honest population sharing one is stated above and on the security page rather than claimed away. `make bench-fairness` measures four bounds, none of them an authentication budget. ## Where this departs from the plan CLAUDE.md changes in two rows only, the two address budgets', which now name the /64; it has no row for `--trusted-proxies` (its table lists variables, and this flag has none), so the flag's new entry is described in `docs/development/configuration.md` and in the flag help. `internal/serverpool` is unchanged: the aggregate reuses `DistinctTokenBudget` rather than adding a type beside it. `cmd/audit_tenancy` needed no new exemption. `docs/development/testing/testing.md` is not regenerated here, as the stack's top regenerates once, so `gen_testing_docs --check` reports this layer's renamed and added tests until then. ## How I tested it Tests first. On the parent the new unit tests do not build (`undefined: KeyNetwork`, `undefined: AuthNetworkFailingPrefixes`, `unknown field NetworkAggregate`, `undefined: mcpotel.AuthBlockNetworkAggregate`). I then ran the four new transport tests from `test/e2e/http` against the parent's binary, and all four fail: `forty addresses rotating inside one /64 were never cut off`, `a /64 the /48 never used = 401 once 10 of its /64s had a credential refused, want 429`, `the server never became healthy` for a socket with `--trusted-proxies=unix`, and neither refused pairing stopping with a refusal that names both flags (the socket one started). On this branch they pass at both doors, beside `TestGate_FailureBudgetIsPerAddress` and `TestGate_RotatingProxyHeader_IsBoundedByTheTransportSource` unchanged. Review added three unit tests, each red on the parent with the one helper it needs copied in: the bearer guard's twin of the gate's unix-socket test (on the parent, with `unix` trusted, the second caller's valid token is answered 429 and never verified), the host guard's test asked through `permits` on a guard bound to a host of its own (on the parent, a unix-socket connection with `unix` listed has its undeclared Host refused), and a `main` row for the socket pairing (the parent tries to bind the path and stops with `its directory is not usable`, naming neither flag). The rewrite of `TestListener_UnixSocket_SharesOneAuthenticationBudget` sits in the commit that adds the startup refusal it would trip, so the transport module passes with that commit's own binary as well as at the tip. The unit tests cover `budgetKey` (whole IPv4, IPv6 /64, mapped, zone, `unix`), `networkKey`, the unix-socket peer on three spellings of `RemoteAddr`, `unix` in `--trusted-proxies` and `trustsPeer`, the host guard's exception through `permits`, rotation inside one /64 spending one fast budget and one distinct-credential record at both doors, ten /64s of one /48 tripping the aggregate and nine not, a credential-less /64 and an IPv4 /24 never moving it, both unix-socket cases at both doors, the aggregate staying on with every other budget off and counting in the configured windows (synctest), the four-budget `longestAuthBlock`, the telemetry count, both startup refusals through the validator, through `main` and on the binary, and the register pins. Results: `go test ./cmd/server ./internal/tenancy ./internal/mcpotel ./internal/config` pass; statement coverage 100% on all four; gobco 2300/2300 on `cmd/server`, 450/450 on `internal/tenancy`, 288/288 on `internal/mcpotel`, 424/424 on `internal/config`; `make coverage-mutants` killed 145 with 0 lived and 0 not covered on `internal/mcpotel`, killed 252 with 0 lived on `internal/tenancy` (its 65 not covered are the `N * time.Minute` initializers of `values.go`, the constant-initializer artifact `docs/development/testing/README.md` describes, none of them a line this changes), and killed 230 with 0 lived on `cmd/server` restricted with `--exclude-files` to five of the six non-test files this changes there (`auth_blocks.go`, `auth_gate.go`, `bearer_guard.go`, `client_ip.go` and `host_guard.go`; a whole-package run costs the 44-second suite per mutant), whose 8 not covered are the same artifact in constants of `auth_gate.go` this does not touch. `main.go`, the sixth, is left out for its size; its one new condition, the `validateTrustedProxyListener` check in `runHTTP`, negated by hand fails seven tests, `TestMain_HTTPFlags_EachRefusalNamesTheFlagThatCarriedTheValue` and `TestRunHTTP_HandsTheToolLayerItsSettings` among them, and the rest of what it changes is flag help and the wiring of the new budget, which `TestRegisterMCPHandlers_SweepEachExpiringTableTheyBuild` and the transport tests exercise; `make check-tenancy` 88 rows, 0 findings; `make check-e2e-static`, `check-test-goroutines`, `check-test-subtests`, `check-test-file-names`, `check-dead-consts`, `check-doc-tool-names` and `check-action-ids` pass; `make test-e2e-http` 678 tests at the tip and 672 with the feat commit's own binary, the 8 Docker-backed ones skipped where no daemon runs; `make test-e2e-stdio` 234 tests; `golangci-lint` clean on the touched packages under the `e2e`, `httpe2e` and `stdioe2e` tags; `go test -tags e2e -c` compiles `test/e2e/gitlab/...`; the site's `i18n:check`, `facts:check`, `chips:check`, `llms:check`, `build` and `lint`, `scripts/check-doc-links.mjs`, `format_md_tables --check`, markdownlint and the em dash check pass. I then rebased the stack onto `main` at `a0e4e6cf6`, with no conflict and every commit's patch unchanged, so the results above were measured on the branch as it stood on `19746966465`. At this head `go build ./...`, `go vet`, `golangci-lint --build-tags e2e` and `go test` on the four packages pass, each at 100.0% of statements; `make check-tenancy` reports 88 rows and 0 findings; `make test-e2e-http` passes; `make test-e2e-stdio` passes with 245 tests, the eleven more than above being the ones `main` added to `elicitation_declared_version_test.go` (#1469); and `make check-closed-servers` and `make check-godocs`, the two gates `main` gained since (#1474, #1478), pass. Of the packages this layer touches only `cmd/server` moved on `main`, in two test files (#1472), so the gobco figures, which count the conditions of the package's code, stand; I did not repeat the mutation runs. The last commit came down from the layer above once CI ran this one alone: `TestRunMain_TheTree_PassesTheGate` and `TestProductionRegister_IsTheRegisterAsItIs` in `cmd/audit_tenancy` pin what the gate reads off the real register, and `AUB-006` moves two of those figures (95 refusals rather than 93, and a derivation of the `network` key between the transport source and the refused credential), so both failed at this layer's head until the commit that updates them sat here.
…1482) Closes #982. This is the third layer of the authentication-budget stack, on top of the /64 key of #954. It bounds the record the transport-source budget keeps of the pairs it charged, as the decisions of 2026-09-27 and 2026-10-17 on the issue set out, and moves `AUB-002` off F-35. ## What was wrong Behind `--trusted-proxy-header` the transport-source budget (`AUB-002`) charges a transport source once per distinct failing primary key inside its window, and to know which keys it already charged it keeps a map of (source, key) pairs, `transportBudget.charged`. Both halves are the caller's to choose, and nothing but the five-minute sweep removed a pair. A blocked source adds none, but the limiter under the budget records at most 4096 sources (`AUB-004`) and at that cap stops recording new ones. A source arriving then is never blocked by this budget, and the budget went on keeping a pair for every key it failed with. ## What changed `AuthRateLimiter.RecordFailure` now reports whether it counted the failure: true for a new key with room, for a live record counted into and for a lapsed record restarted in place, false for the one failure the cap leaves uncounted, a new key the full table has no room for. `AuthRateLimiter.Limit` returns the count the limiter blocks at. Nothing else reads either. `transportBudget.charge` keeps a pair only when its limiter recorded the failure, so a source the full limiter never recorded keeps no pair at all. That also removes a quieter defect: the pair kept for an unrecorded failure stood in for a charge that never happened, so the same key failing again once the limiter had room was deduplicated against it and went uncounted. It is now charged as a first failure, which `INV-007` allows, since the caller caused it. I moved the limiter call under the budget's own lock, so two concurrent failures of one fresh pair are still charged once, as they were when the pair was stored before the limiter was asked. A source holds at most its limiter's limit in pairs (500 in production, `tenancy.TransportSourceDistinctKeys`). At that bound a fresh key takes the place of the source's own oldest pair and is charged as before, as decided on the issue on 2026-10-17. Stopping at the bound instead would let a source just released from its block fail with fresh keys uncharged while the pairs that blocked it are still held, and reclaiming only the source's own pair never takes another source's (`INV-005`). A key whose pair lapsed is re-opened in place and takes no room. I read the bound off the limiter rather than off the constant so that it is the count the limiter blocks at for every limiter, the tests' small ones included; in production the two are the same value. Keeping a pair only for a recorded failure does not bound the sources by itself, although the decision of 2026-10-17 reads as though it did. The limiter's full table lets a lapsed record go at most an eighth of a window after it lapses and takes a new source in its place, while the budget forgot a pair only at its five-minute sweep (`AUB-005`), so a new table's worth of sources stayed for every window between two sweeps: four tables' worth at the default window, measured below, and more the shorter the window, since `AUB-001` accepts any positive one. So once the budget indexes as many sources as the limiter's table can hold, a new source first makes it forget every source none of whose pairs is live, at most once every eighth of a window, as the limiter sweeps its own full table. Past that many, the sources it holds were all charged within one window and an eighth, each while the limiter held a record of it, and a record stays in that table for a window at least, so they are at most three times what the table holds (12288 sources of at most 500 pairs each), whatever the window. That is the decided bound times three rather than the decided bound itself: holding it to one table exactly would cost a pass over the index for every new source at the ceiling, which is the amplifier the limiter's own sweep is throttled against. The pass reads each source's last charge rather than its pairs, and the pairs of a source it forgets are deleted once. The flat map keeps its key format, the source and the key joined by a NUL, so the tests that read it directly are untouched. Beside it a per-source index of the pairs each source holds and when it was last charged bounds the source's share, finds its oldest pair, and finds the sources with no live pair. Finding the oldest scans at most 500 entries and builds no string, and only a source at its bound pays for it: one never blocked whose lapsed pairs the sweep has not reached yet, one whose block has lifted while the pairs that earned it are still held, live or lapsed and not yet swept, or one charged by a request that passed the gates before the block. The sweep walks the index, drops each lapsed pair from both, and drops a source with its last pair. The comments on `transportFailureLimit`, `transportBudget`, `charge`, `maxTrackedAuthSources`, `AuthRateLimiter`, `RecordFailure` and `Limit` now say what is bounded and what is not, instead of claiming the map cannot grow because a blocked source stops charging. ## The register and the specification `AUB-002` records issue 982 among its decisions, declares `AtCapacity: ReclaimOwnThenRefuse` (the source's own oldest pair for a fresh key, and no place for a source the limiter did not record), names `transportBudget.keepLocked`, `transportBudget.forgetLapsedSourcesLocked` and `transportBudget.reclaimOldestLocked` among its enforcing sites, and carries F-16 alone. `AUB-004` declares `transportTrackedSources`, the budget's alias of its value, which is the index size at which the budget forgets the sources with no live pair. F-35 stays in the list, filed as issue 982's, and joins the answered set `TestDecisions_AFindingNoRowCarries_IsAnsweredByItsIssue` names. The specification's F-35 paragraph now says how it was answered, both bounds and the factor of three included, and the count of answered findings is fifteen. What this leaves alone is the saturation itself: a source arriving once the limiter's table is full is still blocked by neither table. As the issue's second decision says, that belongs to the /48 aggregate of #954 and the capacity rules of #958. `cmd/audit_tenancy/declarations.go` needed no change: the one new package-level value is the declared alias, `RecordFailure` is still called only from the declared charge helpers, and `make check-tenancy` reports 0 findings over 88 rows and 370 declared sites. Validation checklist for the change to `AUB-002`: VAL-001 to VAL-005, VAL-007, VAL-009 and VAL-011 are unchanged (Admit, keyed on the transport source, class A, a budget rather than a share, no process partner, the same 429 refusal, the constant 500 whose zero no operator sets, and nothing on stdio, where the budget does not exist). VAL-006 is the change: the row now states its capacity behavior and still marks the structure as a table on caller-chosen values, and the index it keeps is bounded across sources by `AUB-004`'s value as above. VAL-008: what is charged is still a judged failure, a credential GitLab refused or one that was missing, and two such failures are now charged where the parent deduplicated them, both the caller's own doing (`INV-007`): the next failure of a key whose earlier failure the full limiter did not record, and a fresh failure of a key whose pair was reclaimed at the source's bound while it was still live. Forgetting a source drops no live pair, so it charges nothing new. VAL-010: telemetry is unchanged, the refusal is still counted under `transport_source` and nothing about a source or a key is recorded. VAL-012: the change claims a memory bound rather than protection of one population from another, so the measurements are the issue's own and the rotation the cross-source bound answers, below. ## How it was tested The tests came first. Eleven new tests in `cmd/server/auth_gate_test.go` pin the bounds, and nine of them read only the flat map, so they compile against the parent as it stood before the stack was rebased onto `main` (`c1835d5b2`, now `2dbd166fd`) and fail there: - `TestTransportBudget_Charge_KeepsNoPairForASourceTheFullLimiterDidNotRecord`: 4096 sources fill the limiter, then one more fails with 100000 keys. Parent: 100000 pairs for it, 104096 in all. Now: none, 4096 in all, and the source is not blocked. - `TestTransportBudget_Charge_BlocksAtTheLimitAndKeepsNoMoreThanItsPairs`: blocked at 500 keys as before, then 500 more. Parent: 1000 pairs. Now: 500, and another source's pair is kept. - `TestTransportBudget_Charge_AfterTheBlockLifts_StillChargesAFreshKey` (synctest): after the block lifts with one of its two pairs still live, a fresh key reclaims the source's own oldest pair and the next one blocks it again. Parent: 3 and then 4 pairs. - `TestTransportBudget_Charge_ALapsedPairReopensInPlace` (synctest): re-opening a lapsed pair keeps both pairs, and the next fresh key takes the oldest. Parent: 3 pairs. - `TestTransportBudget_Charge_ChargesAFailureOnceTheFullLimiterHasRoom` (synctest): an unrecorded failure keeps no pair and blocks nothing, and the same key is charged once the limiter has room. Parent: the pair is kept and the later failure goes uncharged. - `TestTransportBudget_Cleanup_KeepsTheRoomOfAPairItKeeps` (synctest): a pair the sweep keeps still counts against its source's room. Parent: 3 pairs. - `TestTransportBudget_Charge_TurningTheLimiterOverDoesNotStackSources` (synctest): four generations of 4096 new sources, each arriving once the last one's records and pairs have lapsed. Parent: 8192 sources after the second generation. Now: 4096 after each. - `TestTransportBudget_Charge_HoldsAtMostThreeTablesOfSources` (synctest): six generations, each source charged its last key a second before its record lapses, so its pair outlives the record. Parent: 16384 sources after the fourth generation, past the 12288 of three tables. Now: at most 8192. - `TestTransportBudget_Charge_ForgetsLapsedSourcesAnEighthOfAWindowApart` (synctest): a new source at the ceiling forgets the lapsed sources, the next one an eighth of a window less a nanosecond later does not, the one exactly an eighth later does, and a source last charged exactly a window ago is kept. Parent: 4097 sources where the first sweep leaves 4. The tenth, `TestTransportBudget_Cleanup_ForgetsASourceWithNoPairLeft`, reads the per-source index this change adds, so it cannot compile against the parent; it pins that the sweep leaves no empty source behind. The eleventh, `TestTransportBudget_Charge_ChargesAConcurrentFreshPairOnce`, pins what asking the limiter under the budget's lock buys: 32 concurrent failures of one fresh pair, 200 times, block no source whose limit is two. It passes on the parent, which stored the pair before it asked the limiter, and failed five runs out of five (14 to 26 sources of 200 blocked) on a copy of this branch that asks the limiter after releasing the lock. In `internal/serverpool`, `TestAuthRateLimiter_RecordFailure_ReportsWhetherItRecorded` and `TestAuthRateLimiter_Limit_IsTheCountThatBlocks` fail to compile against the parent (`RecordFailure` returned nothing and `Limit` did not exist). In `internal/tenancy`, `TestDecisions_TransportSourcePairs_AreBoundedByIssue982` and the answered-set expectation of `TestDecisions_AFindingNoRowCarries_IsAnsweredByItsIssue` fail against the parent's register. All of them pass on this branch, and `TestTransportBudget_ChargesOncePerKeyPerWindow` and `TestTransportBudget_Cleanup_ForgetsLapsedPairsOnly` pass unchanged. The issue's measurement, run with a throwaway test against the parent and against this branch (never committed). With the limiter's table full, one more source failing with N distinct keys: | N | Map length, parent | Map length, this branch | | ------ | ------------------ | ----------------------- | | 10 | 4106 | 4096 | | 1000 | 5096 | 4096 | | 10000 | 14096 | 4096 | | 100000 | 104096 | 4096 | With the table not full, the source is blocked after 500 distinct keys on both, and the map holds N pairs on the parent (1000, 10000 and 100000) and 500 on this branch. The rotation the cross-source bound answers, measured the same way with a limit of 10 so that the pairs stay small, and inside one sweep interval, since the five-minute sweep resets both. Each generation is 4096 new sources failing with 10 keys each: in the first rotation a generation arrives a window and an eighth and a second after the last, in the second each source fails with its last key a second before its record lapses and the next generation arrives two seconds later. Keeping a pair only for a recorded failure, without forgetting sources, measures exactly as the parent does here. | Generation | Rotation, parent | Rotation, this branch | Late last key, parent | Late last key, this branch | | ---------- | ---------------- | --------------------- | --------------------- | -------------------------- | | 0 | 40960 | 40960 | 40960 | 40960 | | 1 | 81920 | 40960 | 81920 | 81920 | | 2 | 122880 | 40960 | 122880 | 81920 | | 3 | 163840 | 40960 | 163840 | 81920 | Checks, all on this branch: - `go test ./cmd/server ./internal/serverpool ./internal/tenancy ./cmd/audit_tenancy -count=1`: pass, each at 100.0% of statements; `go test -race` over the budget, gate and bearer guard tests: pass; `cmd/bench_resources` and `internal/finegrained`, the other readers of the register and the pool: pass. - `make coverage-conditions` (gobco, `GOBCO_GATE=all`): `cmd/server` 2316/2316, `internal/serverpool` 418/418, `internal/tenancy` 450/450. - `make coverage-mutants` on the files this changes, the others excluded with `GREMLINS_FLAGS=--exclude-files`: `cmd/server/auth_gate.go` 112 killed, 0 lived, and 8 not covered, which are mutations in package-level constant initializers (the two negative error codes and the string concatenations of the two missing-token messages) that carry no statement counter, the tool artifact `docs/development/testing/README.md` describes, and the parent reports the same way; `internal/serverpool/rate_limit.go` 22 killed, 0 lived, 0 not covered; `internal/tenancy/decisions_admit.go` has no mutable expression. - `make check-tenancy`: 0 findings. - `make test-e2e-http`: 678 tests, 8 skipped for want of Docker, and `TestGate_RotatingProxyHeader_IsBoundedByTheTransportSource` and `TestGate_BlockedAddressStillServesAnAdmittedCredential` pass unedited. - `golangci-lint run --build-tags e2e` and `golangci-lint fmt --diff` on the four changed packages, the e2e compile, `check-test-subtests`, `check-test-goroutines`, `check-test-file-names`, markdownlint on the specification, `format_md_tables --check`, `check-doc-links` and `make check-em-dash`: clean. I then rebased the stack onto `main` at `a0e4e6cf6`, with no conflict and every commit's patch unchanged, so the checks above were run on the branch as it stood on `19746966465`. Of the packages this layer touches, `main` moved `cmd/server` in two test files and `internal/serverpool` in its tests and the place of one doc comment (#1472) and `cmd/audit_tenancy` in its scaffolding (#1479). At this head `go build ./...`, `go vet`, `golangci-lint --build-tags e2e` and `go test` on the four packages pass, each still at 100.0% of statements; `make check-tenancy` still reports 0 findings over 88 rows and 370 declared sites; and `make check-closed-servers` and `make check-godocs`, the two gates `main` gained since (#1474, #1478), pass. The gobco figures count the conditions of code `main` did not change, so they stand; I did not repeat the mutation runs.
Closes #1053. This is the fourth layer of the authentication-budget stack, on top of the transport-source bound of #982. It puts the three documents the server publishes without a credential behind one token bucket per client address, as the decisions of 2026-10-06 and 2026-10-17 on the issue set out: on by default with figures I measured for it, `0` to turn it off, one per-address row in the tenant policy register with the finding that records why it has no process partner, one bucket shared by the three documents, preflights counted, `/health` left out, and a fixed `Retry-After` in the register. ## What was wrong The SEP-2127 card (`/server-card`, `/mcp/server-card` and both under the `--public-url` prefix), the legacy enumerating card (`/.well-known/mcp/server-card.json`) and, in oauth mode, the RFC 9728 metadata are answered to anyone, before authentication, and nothing bounded how often one caller could ask. The server-card extension asks a server to rate-limit requests for its card. The legacy card is 141 KB on the dynamic surface, so two addresses flooding it drew 141 MB/s of egress from the process in my measurement, and every request is a full trip through the HTTP stack. ## What changed `cmd/server/discovery_rate.go` adds `discoveryRateLimiter`: a `rate.Limiter` per client address in a table of at most 4096 (`tenancy.DiscoveryTrackedAddresses`), keyed exactly as the authentication budgets key an address (`budgetKey(clientIP(...))`: the peer, or behind `--trusted-proxies` the address `--trusted-proxy-header` names, an IPv6 address by its `/64`). Its middleware charges every request whose path is one of the documents' paths (`discoveryDocumentPaths`, the same paths `serveHTTPOn` mounts the documents at, prefix included), whatever the method, so a `304` revalidation and a CORS preflight cost what a fetch does, and passes everything else untouched. A request the bucket cannot pay for is answered `429` with `Retry-After: 1`, the gate's JSON-RPC body with code `-42900` and the words "Too many requests for the discovery documents from this address. Retry later.", `Access-Control-Allow-Origin: *` and `Retry-After` exposed, so a browser client can read the refusal the way it reads the documents. The refusal is charged to no authentication budget, and it is logged through the same once-a-window throttle the other refusals use, in words that name the limit and not the address. The middleware sits on the root chain, outside the CORS layer and inside host validation. Outside CORS because `corsMiddleware` answers a trusted origin's preflight itself before the mux is reached, so a bucket wrapped around the document handlers would never see the preflights the decision says to count. Inside host validation so that a request refused for its `Host` reaches no document and costs nothing. A test in `cmd/server` pins each side, and the e2e pins the first on the binary, where the `--public-url` origin is trusted with no `--trusted-origins`. A table at its bound gives a new address the place of the one used least recently, and the five-minute sweep the authentication tables run on (`AUB-005`'s interval, whose row now names this table too) forgets every bucket that has refilled. The settings are `--discovery-rate-limit-rps` and `--discovery-rate-limit-burst`, and `GITLAB_MCP_DISCOVERY_RATE_LIMIT_RPS` and `GITLAB_MCP_DISCOVERY_RATE_LIMIT_BURST` through the HTTP env overlay, both added to `prefixedNames` (and given no old spelling, below). Defaults are 20 a second and 200, maxima 1000 and 10000, `0` for either turns the bucket off, and a malformed, negative, NaN or out-of-range value refuses startup with a message naming the flag (`validateDiscoveryRateBounds`, called from `validateHTTPPoolAndRateBounds`). Both are HTTP-only flags, so a stdio run that is given one names it at startup and ignores it, and both are announced in the HTTP startup record, zero included, so an operator who turned the bucket off reads that it is off. ## The register and the specification `RTC-008` is the new row: Allow, a Rate on `KeyAddress` with no stdio key, class A, `Table: true`, `AtCapacity: EvictOldest`, `ProtectsProcess` with no partner, configurable with `RefuseStartup` and `ZeroOff`, decided by issue 1053 and carrying F-36. Its six values are in `values.go` with their pins (`DiscoveryRate`, `DiscoveryBurst`, `DiscoveryRateMax`, `DiscoveryBurstMax`, `DiscoveryTrackedAddresses`, `DiscoveryRetryAfter`), the server and config constants are declared aliases of them, and every enforcing site is declared on the row, which `make check-tenancy` holds: 89 rows, 387 declared sites, 0 findings. F-36 records the decision of 2026-10-17 that no bucket keyed on the process stands beside this one, a departure from `INV-004`: a bucket the whole process shared would let a flood from many addresses deny discovery to every caller, a hosted client's shared egress address first. Recording it needed one change to `Validate`: its `INV-004` check looked only at requester keys, so a per-address row protecting the process with no partner passed without a finding. It now holds a key before admission to `INV-004` as it holds a requester's, since an address is as mintable as a credential, and `RTC-008` passes by F-36. No other row is affected; the tenancy suite says so. `Retry-After` needed a source of its own, `RetryAfterRefill`, a register constant of a bucket's own (`DiscoveryRetryAfter`), because the decision asked for a fixed second rather than one derived from the bucket's state, and `cmd/audit_tenancy`'s declarations name `discoveryRetryAfter` as the read that satisfies it. Two things differ from the plan this layer was written against, and in both the issue and the code won. The plan said `StopCounting` at capacity; I chose `EvictOldest`, read for a bucket as least recently used, since a bucket has no expiry (the enum's comment now says so). Neither rule limits a caller holding more addresses than the table, which any IPv6 `/48` does: under eviction it cycles them and finds each forgotten when it returns, and against a table that stopped counting it would fill the table and then ask from one more address uncounted until the sweep. Refusing newcomers at the bound would close the gap only by denying discovery to every new address while the flood lasts, which is what the missing partner avoids, so the gap is part of what F-36 accepts, and the row, the specification and the HTTP server page say so. What eviction changes is the cost: an address that keeps asking stays counted, so a flood has to keep cycling through more than 4096 addresses, drawing about one burst from each before it is forgotten, where a table that stopped counting would let a few thousand single requests from distinct `/64`s leave every newcomer uncounted until the sweep. Evicting also forgets a record against its own address and takes no holding from anybody (`INV-005`). The plan said to wrap the document handlers; that would miss the preflights CORS answers first, as above. ## The old spellings Adding the two variables to `prefixedNames` made `RetiredEnvUses` look for their bare suffixes and report a `DISCOVERY_RATE_LIMIT_RPS` in somebody's shell as "no longer read (removed in 3.1.0)", a name no version ever read. The same was already true of the four authentication budgets, which became settings in 3.1.0 after that release had stopped reading old spellings. The last commit makes the settings with an old spelling a closed list, `hadOldSpelling`, the 28 that 3.0.0 read, held by a test to that count and to the settings the server still defines, so a setting added later needs nothing for its bare suffix to go unreported. The environment reference now says 28 settings and 22 bare names, in English and Spanish, and names the six that have none. This changes behavior for the four authentication budgets too: a bare `AUTH_FAILURE_LIMIT` no longer produces the warning, which was wrong for them. Validation checklist for `RTC-008`: - VAL-001: Allow. - VAL-002: `KeyAddress`, the authentication budgets' key, with its mint cost as the register already states it; no new key. - VAL-003: class A, stated and reason unit the process. - VAL-004: a ceiling on a rate, not a share. - VAL-005: it protects the process and has no process partner, by issue 1053's decision, recorded as F-36. - VAL-006: `EvictOldest` over a table of 4096, least recently used, for the reason above, with the gap past 4096 addresses recorded under F-36. - VAL-007: the gate's `429`, code `-42900`, `Retry-After: 1` from the register, the words above, readable from any origin; on stdio the documents are not served and the row has no key. - VAL-008: charges nothing; no authentication budget sees a discovery refusal. - VAL-009: 20 a second and 200, maxima 1000 and 10000, configurable, `0` turns it off, a malformed value refuses startup. - VAL-010: no new telemetry. The middleware sits inside `mcpotel.ServerMiddleware`, so a refusal reaches the HTTP server span and metric as a 429 with its method, like every other gate refusal (that layer records no route or path, by design), and the log line carries the limit's two figures and nothing else, never the address, which a test holds to that exact set of attributes. - VAL-011: nothing on stdio, where no document is served; the two flags are named and ignored there. - VAL-012: the measurement below. ## The measurement (VAL-012) `cmd/bench_resources` gains a `discovery` bound for `make bench-fairness BOUND=discovery`: document verbs (the card, its revalidation with a tag read before the arm, the legacy card, the metadata), each a row of its own; a flood from one address per noisy lane; and a quiet population that is a hosted client's egress address, eight lanes sharing one address, each a user fetching the metadata and the card and then calling `tools/call`. The bucket is per address, so the plan's arithmetic counts what one bucket is offered rather than what one lane offers. `BenchmarkDiscoveryDocuments` in `cmd/server` prices one request through the real handler chain. I ran both on a 16-thread i5-14400 in a container held to 5 cores: - One request is a round trip of 40 to 95 microseconds whatever the document, dominated by HTTP itself; a refusal costs about what the metadata does. The SEP-2127 card is 1,371 bytes, the legacy card 140,934, the metadata 288, a refusal 132. - Sixteen addresses flooding at 100 a second each: 0.25 and 0.30 cores busy without the bucket, 0.26 and 0.22 with it; every flooding address refused four requests in five with it in force, each document row served its share; the shared egress address refused nothing in either arm; the quiet tenant's `tools/call` at 10 to 15 ms at the median in both arms. - Two addresses at 2,000 a second each: 0.42 to 0.47 cores without the bucket, 0.37 with it, most of what remains being the refusals themselves (about 92 microseconds each against about 110 served); the legacy card's egress falls from 141 MB/s to about 1.6 MB/s; the shared egress address refused nothing. - The verdict of every run is `indistinguishable`, for the harness's saturation reason: discovery requests at the rates one address can be held to do not contend for the machine, so what the bucket bounds is each address's draw, not the quiet tenant's latency. That sized the defaults. At 20 a second one address costs the process at most about 2.2 ms of processor a second and 2.8 MB/s of egress on the legacy card. A hosted client's shared egress address has 72,000 discovery requests an hour, so a client that fetches the metadata and the card on every connection connects 36,000 users an hour from one address before its bucket runs dry. A burst of 200 lets a reconnect wave of 100 users from one address through at once. `Retry-After` is a fixed second, inside which a bucket refilling at 20 a second, or at any rate of one a second or more, has a request to give again; at a configured rate below one a second it is shorter than the refill, which the HTTP server page says. 4096 addresses is the authentication tables' figure, a few hundred kilobytes. The first run used four times the rate per address, and it showed an artifact of the open-loop schedule rather than of the bucket: a lane cycles its four verbs on fixed ticks, a token refilled every fourth tick, and two of the four documents got every token. The bound now defaults five times the rate, and a test holds the ticks per token and the verb count to sharing no factor. ## Documentation The site's HTTP server page gains a section on the bucket in English and Spanish, and the CLI, environment, configuration, remote deployment and security pages name the two settings and what the address key means behind a proxy: behind a proxy the server is not told to believe, every caller shares the proxy's one bucket, so either name the real address or turn it off. The HTTP server page also says what the fixed `Retry-After` means below one request a second, and that a caller cycling more addresses than the table holds is held back by no bucket, so a deployment exposed to that limits these paths at its proxy as well. `docs/development/configuration.md` lists both settings and the closed list of old spellings, the specification records `RTC-008` and F-36, `docs/development/cmd-utilities.md` and the `bench-fairness` comment describe the new bound, and `CLAUDE.md` gains only the variables' row. ## How it was tested The tests came first, and every new one is red on the parent as it stood before the stack was rebased onto `main` (`d14bb760c`, now `5db3a1700`), with this layer's test files copied onto its tree: `cmd/server`, `internal/config`, `internal/tenancy` and `cmd/bench_resources` do not build (`discoveryRateLimiter`, `DiscoveryRateLimitRPS`, `RTC-008`'s constants and `documentRequest` are undefined); `cmd/audit_tenancy` fails `TestRunMain_TheTree_PassesTheGate` on the gate's summary; the four `TestDiscoveryRate_*` cases of `test/e2e/http` fail against the parent binary (it refuses the two flags at startup, announces no bucket and serves every fetch), and the stdio case fails on `flag provided but not defined: -discovery-rate-limit-rps`. - `cmd/server/discovery_rate_test.go`: the burst and its refill, addresses kept apart, the least recent address evicted at the table's bound, the sweep forgetting refilled buckets, zero turning the bucket off, the document paths per mode and prefix, only the document paths charged (a fetch, a preflight and a HEAD from one bucket; `/health`, `/mcp` and unserved paths untouched), the refusal's status, body and headers, its log line held to the limit's two figures and no other attribute, the key through a trusted proxy and by `/64`, and `serveHTTPOn` wired end to end in both auth modes under a prefix (200, then 304, then a refused preflight from the trusted public origin, with `/health` and `/mcp` still answered), a request with an undeclared `Host` at every document path refused 403 and charged nothing, plus the bounds validation table. The placement tests were shown red on mutants of `serveHTTPOn`: with the bucket inside the CORS layer every preflight is answered 204, in the unit test and in the e2e alike, and with it outside host validation the 403s spend the bucket and the next good fetch is refused. A log line given the address fails the attribute test. - `internal/config`: `hadOldSpelling` held to the 28 settings 3.0.0 read and to the settings the server defines, and `RetiredEnvUses` reporting nothing for the bare suffixes of the four authentication budgets and of the two discovery settings, red on the old loop over every setting. - `cmd/server` flags, help, startup record and env overlay, `internal/tenancy` (the row, its pins, F-36, the widened `INV-004` check) and `cmd/audit_tenancy`. - `test/e2e/http/discovery_rate_test.go` drives the real binary: each document at every path under a prefix in both modes, the forwarded address believed only from a trusted proxy, `0` through the flag and the variable, and the shipped default refusing one address past its burst of 200. `test/e2e/stdio` names the two flags as ignored on stdio. - `make test-e2e-http`: 703 tests, 8 skipped (the ones that need Docker), none failed. `make test-e2e-stdio`: 246 tests, none failed (235 before the rebase below; `main` added eleven, #1469). - `make check-tenancy`, `check-dead-consts`, `check-test-goroutines`, `check-test-subtests`, `check-test-file-names`, `check-doc-tool-names`, `check-action-ids`, `check-e2e-static`, `check-gateway-chars`, `check-sdk-context` and `check-md-escaping` pass, the `e2e` suite compiles, and `golangci-lint` with the `e2e`, `httpe2e` and `stdioe2e` tags reports nothing on the touched packages. - Statement coverage is 100% and condition coverage (gobco) complete on every touched package: `internal/tenancy` 454/454, `internal/config` 436/436, `cmd/audit_tenancy` 994/994 (1016/1016 before the rebase below, the 22 fewer being in the scaffolding #1479 moved out of the package), `cmd/server` 2356/2356, `cmd/bench_resources` 1592/1592. - `make coverage-mutants` on the files this changes, the others excluded with `GREMLINS_FLAGS=--exclude-files`, 0 lived and 0 timed out everywhere: `cmd/server` (`discovery_rate.go`, `main.go`, `env_overlay.go`, `http_only_flags.go`) 434 killed; `internal/config` (`config.go`, `http_overlay.go`, `env_name.go`) 176; `internal/tenancy` (`decision.go`, `validate.go`, `decisions.go`, `decisions_allow.go`, `decisions_admit.go`, `values.go`) 185; `cmd/audit_tenancy` (`declarations.go`) 22; `cmd/bench_resources` (`fairness.go`, `fairness_run.go`, `fairness_result.go`, `presenter.go`, `rpc.go`) 465. The not-covered mutants (16, 9, 41, 5 and 9) are all in package-level constant and variable initializers, which carry no statement counter, the tool artifact `docs/development/testing/README.md` describes. Two of them are lines this layer adds, `DiscoveryRetryAfter = 1 * time.Second` and the discovery bound's defaults, and the tests pin both values; every other one is on a line this layer does not touch. - The site builds and lints, `i18n:check`, `facts:check`, `chips:check` and `llms:check` pass, the Markdown tables are formatted, markdownlint is clean and `scripts/check-doc-links.mjs` finds every link and anchor. I then rebased the stack onto `main` at `a0e4e6cf6`, with no conflict and every commit's patch unchanged, so the results above were measured on the branch as it stood on `19746966465`, except the two figures that say otherwise. Of the packages this layer touches, `main` moved `cmd/server` and `cmd/bench_resources` in their tests and by one blank line in `fairness.go` (#1472) and `cmd/audit_tenancy` in its scaffolding (#1479). At this head `go build ./...`, `go vet`, `golangci-lint --build-tags e2e` and `go test` on the five packages pass, each still at 100.0% of statements; `make check-tenancy` still reports 89 rows, 387 declared sites and 0 findings; `make test-e2e-http` and `make test-e2e-stdio` pass; gobco on `cmd/audit_tenancy` is the 994/994 above; and `make check-closed-servers` and `make check-godocs`, the two gates `main` gained since (#1474, #1478), pass. I did not repeat the mutation runs.
Closes #1102. This is the fifth layer of the authentication-budget stack, on top of the discovery documents' bucket of #1053. It changes the words of the gate's session refusal and nothing about when it refuses, as the decision of 2026-10-17 on the issue set out: one answer for both cases, no record of each credential's ended session IDs, an INFO line of its own for a session the process does not hold, and the WARN line for another credential's session kept. ## What was wrong On `--stateless=false` the gate answers `404` to a request whose `Mcp-Session-Id` names no session open for the credential it presents, and the answer said "This session does not belong to the presented credential." That is true of a session another credential opened. It is false of the case a client meets every day, its own session that idled out past `--session-timeout`, was deleted, went with its credential's pool entry, or was opened before a restart or on another replica: nothing that client presented belongs to anybody else, and the message told it otherwise. Row ADM-007 of the tenant policy register described only the first case too. ## What changed `sessionOwnershipFailure` in `cmd/server/auth_gate.go` now answers "This session is not open for the presented credential. Start a new session by sending initialize without a session ID.", which is true of both cases. The status, the code (`-32600`) and when it is answered are unchanged. I kept one answer rather than two because answering the cases apart would tell a caller whether another credential holds the ID, which INV-019 forbids, and because it would need a record of each credential's ended session IDs, state keyed on a value the caller mints, with a row and a capacity of its own. `checkSessionOwnership` tells the cases apart in the log instead. Another credential's session keeps its WARN line, `request rejected: session ID does not belong to the presented credential`, word for word, since it is true of that case and an operator may already alert on it. An ID the process does not hold gets an INFO line, `request rejected: session ID names no session open in this process`, through the same once-a-minute throttle as the gates' other refusal lines, since a caller can send any ID it likes. Neither names the ID. Row ADM-007 in `internal/tenancy/decisions_admit.go` moves with the gate: its resource is now "a session ID that names no session open for the presented credential", its refusal prefix is the new first sentence, and it records issue 1102's decision. It stays a rule with no value, no table and no capacity, which is the half of the decision a test holds it to. The e2e phrase the go-sdk client pins expect (`test/e2e/http/session_not_found_test.go`) and row 68 of `docs/development/upstream-bugs.md` quote the new words. Row 68 no longer says the call path shows the wrong cause; what a go-sdk v1.8.0 client is left with there is a true refusal on a connection that looks alive, with an error that does not wrap `ErrSessionMissing`. The HTTP server page, in English and Spanish, says the `404` answers both cases alike, why, and which log line names each, and its refusal table now names that row's condition as ADM-007 does, "an `Mcp-Session-Id` that names no session open for the presented credential", where it said one "this deployment did not open", which is false of the client's own session that ended. A last commit puts `resolve`'s doc comment back on `resolve`. It had drifted above the `Mcp-Session-Id` header constant beside the check this changes, so godoc read it as part of the constant's comment and `resolve` had none. ## How I tested it Tests first, each shown red on the parent as it stood before the stack was rebased onto `main` (`3df394937`, now `b37e82899`) and green on this branch: - `TestDecisions_EachRowIsWhatTheSpecificationSays/ADM-007` pins the new prefix, and `TestDecisions_SessionRefusal_IsOneAnswerByIssue1102` pins the resource, the recorded decision, the absence of any value, table or capacity, and the one refusal. - `TestMcpServerGate_CheckSessionOwnership_RefusesASessionFromAnotherCredential` now holds every refusal (another credential's session, an ID never recorded, a session whose credential was evicted) to the same code and the same full text, and asserts the log: the WARN line for the first, the INFO line for the other two, and neither for a request the check lets through. On the parent the text differed and the INFO line was missing. - `TestMCPServerGate_EachRefusal_CarriesItsOwnStatusAndCode` drives a real foreign session (its old case was in fact an unrecorded ID) and an unrecorded one through the middleware. Since both are answered alike, each also asserts the line the session check logged, WARN for the foreign session and INFO for the unrecorded one, and every other case that it logged neither; without that, a foreign session whose ID was never recorded passes as one, which is how the old case went unnoticed. I checked that both ways: with the foreign case's ID swapped for an unrecorded one, the table without the log assertion still passes and this one fails on the log line. `TestMcpServerGate_Middleware_RefusesASessionMintedForAnotherCredential` holds POST, GET and DELETE to the full text. - `TestGate_SessionRefusal_IsOneAnswerOnTheWireAndTwoLinesInTheLog` in `test/e2e/http` drives the binary: another credential's session and an ID the process does not hold are answered with byte-identical bodies, and the server's stderr carries the WARN line for the first and the INFO line for the second. The two go-sdk pins of row 68 pass with the new phrase. Results on this branch: `go test ./internal/tenancy/... ./cmd/server/... ./cmd/audit_tenancy/...` passes; `make check-tenancy` reports 0 findings over 89 rows; `make test-e2e-http` passes, 708 tests with 8 skipped for want of Docker; statement coverage is 100.0% for `cmd/server` and `internal/tenancy`; gobco conditions are 2356/2356 for `cmd/server` and 454/454 for `internal/tenancy`; `make coverage-mutants` over the whole of each package, no file excluded, reports Lived 0 and Timed out 0 on `cmd/server` (1301 killed, the five in `checkSessionOwnership` among them, in 3 hours 7 minutes) and on `internal/tenancy` (255 killed). The 47 and 66 it calls not covered are the tool artifact `docs/development/testing/README.md` describes, mutations in package-level constant and variable initializers, which carry no statement counter (the gate's error codes and message concatenations, the HTTP timeouts and other durations, the env flag names, the listen endings' and subscription errors' texts, the register's codes, values and `iota` enumerations), plus two in `socket_mode_windows.go` and `conn_refused_windows.go`, which are not compiled on Linux; none is on a line this branch touches; `golangci-lint` is clean on both packages under `e2e` and on `test/e2e/http` under `httpe2e`; `go test -tags e2e -c` compiles `test/e2e/gitlab/...`; `make check-test-subtests`, `check-test-goroutines`, `check-test-file-names`, `check-action-ids`, `check-dead-consts`, `check-doc-tool-names`, `check-md-escaping` and `check-e2e-static` and the godoc audit pass; markdownlint, the Markdown table check, the doc link check, and the site's i18n, facts, chips, llms, build and lint checks pass. I then rebased the stack onto `main` at `a0e4e6cf6`, with no conflict and every commit's patch unchanged, so the results above were measured on the branch as it stood on `19746966465`. Of the packages this layer touches, `main` moved `cmd/server` in two test files (#1472). At this head `go build ./...`, `go vet`, `golangci-lint --build-tags e2e` and `go test` on `cmd/server` and `internal/tenancy` pass, each still at 100.0% of statements, and so do the tests of `cmd/audit_tenancy`, also at 100.0%; `make check-tenancy` still reports 0 findings over 89 rows; `make test-e2e-http` passes with the same 708 tests; and `make check-closed-servers` and `make check-godocs`, the two gates `main` gained since (#1474, #1478), pass. The gobco figures count the conditions of code `main` did not change, so they stand; I did not repeat the three-hour mutation run. ## Tenant policy checklist This changes a refusal's text and a row's resource, not a limit. VAL-001 to VAL-006, VAL-008, VAL-009 and VAL-011 are unchanged on ADM-007 (Admit, keyed on the session, class C, a rule, nothing kept, not charged, no value, none on stdio). VAL-007 is the change: the gate's `404` with `-32600`, no `Retry-After` and no challenge, answer "start over", now in words true of both cases. VAL-010: with telemetry on, the new INFO line is exported as a log record, as the WARN line beside it already is; it carries no attribute but the throttle's count of repeats it held back, so it names no session ID and no credential, and no metric, span or reason changes. VAL-012 does not apply, since nothing here protects one population from another. The generated testing reference gains the new tests' entries when the top of the stack regenerates it, as the stack's other layers leave theirs.



This is the first layer of #1132: the helper and the sites. The gate that keeps the pattern out comes in the next layer, and the Makefile, CI and documentation wiring in the one after it.
A test that needed an instance that is gone started an
httptestserver and closed it, keeping the URL. Closing it gives the port back to the kernel at once, and whatever is given that port next can answer the request the test exists to see fail: a parallel test of the same package starting a server of its own, or a test binary of another package running beside it in the same coverage run. #1131 fixed one such flake incmd/bench_resources; this moves every other site.internal/testutil/netfixturehas two things.DeadEndpoint(t)holds a loopback port for the whole test and closes every connection it accepts before reading a byte, and its cleanup waits for the accept loop to end. An accept that fails for any reason but the listener being closed, such as a process out of file descriptors while several test binaries run at once, is retried after a pause, asnet/http's server does: a loop that ended there would keep the port with nothing accepting, and the kernel would queue every connection made to it, so a request would wait out its deadline instead of failing at once.NewStoppableServer(t, handler)answers through its handler untilStop, then hangs up on every request, the ones on a connection a client kept from before included, closes the connections already open, and keeps its port. Each gives its port back in the cleanup it registers, which runs after every cleanup registered later and before any registered earlier, so a cleanup registered before either is asked for must not use its address; their doc comments say so. The bench package'sclosedEndpointmoves ontoDeadEndpoint.The issue asked for the helper in
internal/testutil, but the code does not allow it:internal/testutilimportsinternal/gitlab, sointernal/gitlab's own tests, which hold four of the sites, cannot import it (import cycle not allowed in test). The package sits besidetestutilinstead, the wayinternal/testutil/hintsdoes for the same reason, imports nothing of this module, and is added to the test supportTestDependencies_TestSupport_NeverReachesTheServerBinarykeeps out of the server binary.The sites are the sixteen the gate in the next layer finds on this branch's parent, in seven packages. Fourteen are the issue's own rows that still exist, at the line numbers they have moved to: the issue lists fifteen, and
TestCredentialRejected_OnlyAnExplicitRefusalCountsAsOnewent away with #1139, so the issue's three credential tables ininternal/gitlabare two now (TestCheckCredential_ThreeAnswers_KeepsNoVerdictApartisTestCheckCredential_FiveAnswers_KeepsEachApart). The other two arrived with #1146, after the issue was opened:TestAdmitToken_AssumedScopes_SaySoInTheAdmissionininternal/oauthandTestDetectToken_NoAnswer_IsNothingKnownininternal/gitlab/scopes_test.go. One more of the issue's rows changed shape before this:TestRunProbe_ExplicitTarget's "nothing listening" case had been moved into a sequential test of its own (#1258) because the reset it could meet broke itsrefusedassertion. I read each site before changing it.As decided on the issue on 2026-10-17, the probe stops asserting
refused: an accepted and closed connection ends with an EOF or a reset. The case goes back into the parallel table on a dead endpoint and holds what the probe is for, an unhealthy exit that names the target that gave no answer, and the sequential test is gone. The comments that described a refused connection ininternal/oauth,internal/gitlabandcmd/internal/apidocsnow say what happens.TestNewGitLabVerifier_NetworkErrorWithCacheis the one site that needs an instance to answer first and then stop, with the verifier and the cache bound to its URL, so it usesNewStoppableServer. Its comment claimed a failed re-validation removes the cached entry, but the test deletes the entry itself, standing in for its expiry, since a cached identity is answered without asking the instance. The comment now says what the test holds: a re-verification against an instance that stopped answering admits nothing and caches nothing. The two credential tables ininternal/gitlabbuilt their client on a live server and closed it for the unanswered row; building a client sends nothing, so that row now builds it on the dead address from the start, through a smallcredentialEndpointhelper, andinternal/serverpooldoes the same for its unreachable row.Three packages whose tests this layer touches were below the bar on
main, and five commits close them.cmd/audit_graphql_documents(97.0% of statements) ranmainonly in a child process, whose coverage counters are not the test's:mainnow hands its arguments, the environment and both streams torunMainand ends the process throughexitProcess, and two tests in the test process cover both; the child-process tests stay.test/e2e/modeleval/internal/provider(96.4%) had failure paths held for one adapter and not the others, a few cases no test built, and four branches no run can reach: those are reached through the catalog builder handed tobuildFakeCatalog, theencodeCallvariable and the cached catalog error, or removed where the code could not fail (a second decode inproducedValue, an encoding error innormalizeSchema), and its unknown-provider test now holds the refusal it is named for. At 100% of statements it still had twenty conditions evaluated one way only and 28 mutants lived and 21 uncovered, all in code this layer did not otherwise change; a commit of its own answers each of them, with tests for what nothing held (the echo checks of all three adapters, a call named for its position, the wrong-args variant planting one fact, an individual tool name a sibling took, every side of the HTTP status classification and of the price window, the latency in milliseconds, the default client's timeout), four switches with no tag rewritten as ifs because gremlins cannot see into their case expressions, and two checks removed that could never decide anything.cmd/internal/apidocshad a loop condition that was never false, since the loop always left through the break after its last attempt, a status classification in that loop written as a switch with no tag, whose six case mutants gremlins reported uncovered although each dies when applied by hand, and a const group whose one exported name go doc read as undocumented, so the godoc audit reported it. One commit fixes the loop and rewrites the classification as a chain of ifs gremlins can measure, and another the const group. The testing README's account of those six mutants now says so.The godoc audit also reported, in packages this layer touches, three comments go doc does not read where they stand, none of them new here: a second package comment in
cmd/bench_resources, becausefairness.go's file comment sat directly on its package clause;StartIdleEviction's comment ininternal/serverpool, run intorecoverSweep's two functions above it, which left the method undocumented; and, under--include-tests, eleven tests ofcmd/internal/apidocs, the one this layer moves among them, and three ofinternal/serverpoolwith no doc comment at all. The last commit puts each comment on what it documents.How I tested it. The netfixture tests were written first and failed to build on the parent (
undefined: DeadEndpoint,undefined: NewStoppableServer), and the accept-loop test failed on the loop that ended at the first failed accept (read pipe: i/o timeout, want the EOF of a hang-up). They pass now, 50 times in a row under-race. They hold that every request to a dead endpoint ends without an answer and promptly, that its port cannot be listened on while the test runs and is free once it ends, that the accept loop survives a failed accept and still ends once the listener is closed, that a stopped server hangs up on a kept connection and on a new one and is never reached again, thatStopcuts a response being streamed, and that a listener that cannot be opened fails the test naming why. The package is at 100% of statements, 10/10 conditions under gobco, and 4 mutants killed with none lived or uncovered. The newrunMaintests failed to build on their parent (undefined: runMain,undefined: exitProcess);cmd/audit_graphql_documentsis now at 100% of statements, 154/154 conditions, and 88 mutants killed with none lived or uncovered.test/e2e/modeleval/internal/provideris at 100% of statements, 406/406 conditions, and 235 mutants killed with none lived; the two it reports uncovered are the operators of the constant180 * 24 * time.Hourthat sizes the price window, which carries no statement counter, and the price tests hold that window to 180 days at its edge and past it.cmd/internal/apidocsis at 100% of statements and 110/110 conditions, and with the switch gone its mutants went from 62 killed, 4 lived and 10 uncovered to 67 killed, 4 lived and 4 uncovered, every survivor of a kinddocs/development/testing/README.mdrecords under "Reading a survivor": the four lived are the three boundaries whose sides agree and the clock boundary that page names for this package, and the four uncovered are operators in constant initializers. The packages this layer touches only in their tests are at the bar too.internal/oauthis at 224/224 conditions and 114 mutants killed with none lived or uncovered;internal/gitlabat 708/708 and 359 killed, none lived and 8 uncovered;internal/serverpoolat 418/418 and 215 killed, none lived and 6 uncovered. The fourteen uncovered are operators in package-level constant and variable initializers, such as30 * time.Secondandconfig.EnvPrefix + "ALLOW_PRIVATE_INSTANCES", which carry no statement counter: the tool artifact the same page records.cmd/serverandcmd/bench_resourcesare at 2270/2270 and 1544/1544 conditions. This layer changes only their tests (and a blank line infairness.go), and a mutation run of either whole package means thousands of mutants each running the package's whole suite, so I measured the files those tests exercise:cmd/server'sprobe.gohas 83 mutants killed, none lived and 2 uncovered, the operators of its two timeout constants, andcmd/bench_resources'sessions.goandheld.gohave 134 killed with none lived or uncovered. Every moved test passes under-race, every touched package passes in full under-raceand is at 100% of statements, andgolangci-lintwith every e2e tag is clean on all of them. The gate in the next layer, run on this branch's parent, reports exactly these sixteen sites and exits 1; on this branch it exits 0.Refs #1132