Repository navigation
Bound the calls the process holds open at once, sized by its descriptor limit - #1088
Conversation
There was a problem hiding this comment.
Sorry @jmrplens, your pull request is larger than the review limit of 150,000 diff characters
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 59 minutes. View limit detailsLimit details: You’ve used the included review currently available. Review configuration: ⚙️ Run configurationConfiguration used: Repository: jmrplens/gitlab-mcp-server/.coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (64)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The fix for #945 (#948) removed the workflow's own `actions/cache` step for `~/.cache/golangci-lint` and left a second cache of that directory in place: `golangci-lint-action` keeps one itself. `install-only: true` returns after the install and nowhere else, so the action still restores that cache before installing (`run()` calls `restoreCache` first) and saves it again in its post step, under `golangci-lint.cache-<os>-<interval>-<go.mod hash>` keys. The comment in `ci.yml` said the cache was not restored and the static analysis page that it was not kept between runs, and neither was true. On 2026-09-30 the lint of #1088 to #1094, opened minutes before, reported fifty unused `//nolint:staticcheck` directives on each layer, in files the stack does not touch. Every one of those jobs restored, by exact key, the entry main's push run of b44ab96 had saved at 09:07 UTC: the key names only the OS, a seven-day interval and the hash of `go.mod`, which the stack shares with main, so the analysis of main's tree matched a different tree exactly. With that entry deleted, #1087's second run and a rerun of #1088's commit fell back through the restore key to main's entry of 2026-09-25 and linted clean, and a cold `make golangci-lint` of the stack's tip reports nothing. That is the failure #945 describes, a restored analysis disagreeing with the tree, coming from the cache #948 did not see, and this time from main's entry rather than the pull request's own. `skip-cache: true` turns off both the restore and the save. The comment above the step now says why it is needed alongside `install-only` and what the key leaves out, and `docs/development/static-analysis.md` says the same. The job's header comment no longer says the action caches the linter binary, which it downloads on every run. The Go build and module caches setup-go keeps are unaffected. What it costs, measured on the whole job as #945 asks: the eight pushes to main between 29 September 11:27 and 30 September 09:02 UTC each restored a cache and took 214 to 319 s, the `make golangci-lint` step 190 to 298 s of it; this pull request's run, the only one with the change so far, restored nothing and took 313 s, the step 287 s. All eight restored the entry main saved on 25 September, which the action does not save again while its key still matches, so the analysis they reused was of a tree days older than theirs: it was not buying time the spread between runs can show. Closes #945
28dad9d to
8e73aaa
Compare
Every mode of the benchmark drives requests the server answers as fast as it can, so none of them could say what a request the server holds open costs, or what a process holding too many of them does. Issue 951 needs both to size a ceiling on held requests, so this adds a mode that asks. -held <counts> offers a ladder of gitlab_execute_action calls running project.get, spread round robin across -held-credentials, against a stand-in GitLab that now answers a project read and, while a hold is on, keeps it waiting after its headers. Headers first, because the server gives GitLab a minute to answer with headers and would abandon and resend a read held before them, and a step would measure the retry. A step is sampled once every call is held or has come back, or once that count has been still for three seconds (a process out of descriptors accepts no more connections, so some calls are neither), or after two minutes; the wait reads a clock the tests replace, so each end is held to its instant. The sample is the process's open descriptors from /proc/<pid>/fd, goroutines and live heap from the profiling listener, the resident set, and a /health probe on a connection of its own, since a probe riding an old keep-alive connection would answer for a process that no longer accepts one. Every call is then filed as served, refused (a 503) or failed, and one held call is priced as the step's growth over the idle process. -held-nofile starts the server through /bin/sh and ulimit -n, which sets the soft and the hard limit together: Go raises its soft limit to the hard one at startup, so lowering the soft one alone would be undone before the server served anything. The shell execs the server in its own place, so the pid sampled is the server's. The mode writes bench/held.json, uncommitted, draws nothing, and refuses a -held-json that names the published record, like the fairness mode. Calling gitlab_execute_action exposed a gap in the harness's client: from protocol 2026-07-28 the transport refuses a call whose parameter header is missing, and the tool's schema mirrors its action argument into Mcp-Param-Action. The client now sends it, only where the value exists. The stand-in server learns the same call: it reads the project through the --gitlab-url it is given, checks the header, and answers past STANDIN_HELD_LIMIT with a 503, so the whole ladder runs in the unit tests. parseSteps becomes parseCounts, shared by -clients and -held. Part of #951.
A tools/call holds its POST for as long as the call runs: an hour for a pipeline wait, and any call for as long as GitLab keeps it waiting. The rate limit bounds how often a credential calls, not how many of its calls are still running, and the pool bounds entries, not what an entry holds, so nothing bounded how many requests one credential, or all of them, could keep open. Issue 951 recorded that as finding F-31. Measured with the new held mode of cmd/bench_resources on truenas (the real server, a stand-in GitLab holding every project read): - One held call costs 2.00 descriptors (the caller's connection and the one to GitLab), 6 goroutines, about 51 KiB of live heap and about 190 KiB of resident set, linearly: 4000 held calls took 8010 descriptors, 24013 goroutines and 873 MiB resident. - Started under a descriptor limit of 1024 and offered 1000 calls, the process held 503 and then stopped accepting connections at all, /health among them, until they ended; 442 calls failed with "too many open files" at their dial to GitLab. At 4000 offered, 534 failed. - With this change, under the same limit, 4000 calls offered: 192 held and served, 3808 refused with 503, none failed, 402 descriptors, and /health answered in a millisecond. The same at every step from 250. The ceiling is 192 across the process (tenancy.HeldRequestsPerProcess), sized against a descriptor limit of 1024, the default soft limit on most Linux hosts, beside the other kind of request the process holds open: 512 listen streams at one descriptor, 192 held requests at two, and an eighth of the limit spare for the idle process, /health and connections being refused. It is keyed on the process because only a ceiling keyed on the process bounds the process: a credential is one API call to mint, so a per-caller number multiplies by however many a caller holds. And it is not configurable, for the reason the 512 listen streams and watchers are not: an operator who could raise it could undo the bound. Decisions taken with it: - The slot is taken last in mcpServerGate.middleware, after admission and the session ownership check (PAT-003), so a request refused for its credential, its instance or its session never spends one, and it is given back when the handler returns. - Every POST counts except a subscriptions/listen whose Mcp-Method header says so on protocol 2026-07-28 or later, where the SDK holds the header to the body; the listen ceilings count those. An older client's listen is counted here, since its header is not checked against its body. GET and DELETE are not counted: the default stateless transport answers them 405, and a stateful session's standalone stream is one per session, which stays with HLD-010 and F-31 for the third part. - The refusal is a gate 503 with -50300 and the fixed 30 s Retry-After the upstream refusals already advertise (UpstreamRetryAfter), in words that name no credential: "This server is holding as many requests as it serves at once. Retry later." It discloses one bit about the process, which INV-019 accepts for a process bound. The log line names the scope and the figure. - A held call makes no pool entry busy and delays no eviction: the slot counts held work, never a caller. The register gains row HLD-011 (Allow, Ceiling, process-keyed, valued, not configurable) with its value pin and requirement ID; HLD-010 now describes stateful sessions alone. make check-tenancy: 83 rows, 21 failures and 287 declared sites (77 refusals, 13 reasons), 0 findings. The HTTP e2e module holds the whole of it against the real binary: 192 calls held, the next refused in the register's words with its Retry-After and log line, /health and a listen still answered, and every held call served once GitLab answers. The specification, ADR-0023, the HTTP guide, the references and the site pages in English and Spanish say what the ceiling is and what it is not. Part of #951.
… descriptor limit The first cut of HLD-011 counted POSTs in the gate against a fixed 192, and the review of the layer found both halves wrong. A POST is not a call (truth finding 1, house finding 3, truth findings 3 and 4). Below protocol 2025-06-18 go-sdk accepts a JSON-RPC batch, so one slot carried as many calls as the body held, up to the 4 MiB limit with the rate limit off. A POST on an older revision can carry only a response or a notification, which holds nothing, and refusing it at a full ceiling kept a client's answer to an elicitation from the call waiting on it, so the slots never freed. And a listen on an older revision was counted by the listen ceilings and again here. The calls are now counted in an MCP middleware, where the SDK has read the body: every call that reaches GitLab (MeterFor's tool-call and completion buckets) other than a subscriptions/listen, so each call of a batch takes a slot, a response or notification takes none, and a listen takes none on any revision. It runs before the rate limit and refuses the way the rate limit refuses the same method. On protocol 2026-07-28 or later, where the SDK holds Mcp-Method to the body and refuses a batch, the gate still takes the slot itself after admission and hands it to the call through the carrier, so that refusal is a 503 the SDK never reads, and it now closes its connection (truth finding 5): a refused caller that kept its connection held a descriptor of the limit the ceiling protects, and a hundred credentials reached 1141 to 1280 descriptors that way. 192 was sized for a soft limit of 1024 that no Go process runs under (truth finding 2, house finding 4): the runtime raises the soft RLIMIT_NOFILE to the hard limit before main. On a default systemd service or container that cut what the process can hold to about a twentieth and let one credential's pipeline waits have every other tenant refused for up to an hour. The ceiling is now derived at startup from the limit the process actually has: an eighth spare, one descriptor for each of the 512 listen streams, two per held call. That is 192 under a hard limit of 1024 and 229120 under 524288; where the platform has no limit to read (Windows) it is the figure for 1024. The row is Source Derived, and HeldRequestDescriptors, DescriptorSpareDivisor and FallbackDescriptorLimit are pinned in place of HeldRequestsPerProcess. Every refusal now says only "This server is busy. Retry later.", naming no bound (truth finding 8); the log line names the scope and the figure, and the spec puts the INV-019 question to the maintainer instead of stating the exception on the maintainer's behalf. The slot is taken after admission, which departs from issue 951's request to take the process slot before anything per key; the row, the spec and the gate now say so and what a refused newcomer has spent by then, instead of citing PAT-003 for the opposite order (truth finding 7, house finding 2). Tests: the OAuth mode's wiring has a test of its own, which fails if registerOAuthMCPHandlers stops passing the count (truth finding 6, house finding 1). A held call leaves its entry evictable under POL-003, and a test pins that and that the call outlives the eviction (truth finding 11). The e2e test starts the binary under ulimit -n 1024, so its /health assertion can now fail (truth finding 12). The comment that said a stateful GET was bounded is gone with holdsRequest (house finding 11). Lint: the gosec G115 conversion carries its reason, the header copies use maps.Copy, the nil-carrier case is reached through a lookup, and the e2e test is split into helpers to stay under gocognit's bound. make check-tenancy: 83 rows, 21 failures and 299 declared sites over 202 packages (21 refusal returns, 80 refusals, 13 reasons and 31 settings read); 0 findings. Part of #951.
…nt mutants The review of the held mode found the package short of the mutation gate on lines the mode reaches (house finding 8), and the mode documented only as proposed text for the integrator (house finding 13). The survivors were equivalent mutants the code invited, so the code stopped inviting them rather than the tests pretending to kill them. The two scanners in rpc.go started from a 64 KiB buffer whose size changes nothing but the first allocation, so they now start from none and grow to the longest line. firstLine cut at a newline by hand and capped at 200 by hand, whose index >= 0 and len > 200 boundaries no body tells apart; it now reads strings.Cut and min. withoutConfig sized a slice with arithmetic no test can observe and appended AUTOPILOT last; it now starts from AUTOPILOT. healthWait is time.Minute rather than 60*time.Second, and the held mode's four pauses come from heldWaits, which a test holds to their figures before any test replaces them, so no package-level initializer is left uncovered. The stand-in speaks the ceiling's new words, "This server is busy. Retry later.", so the ladder's refusals are recognised as the real server's. docs/reference/resource-benchmark.md gains the held mode's section and flag table, and make bench-held runs the ladder the ceiling was measured with, under a hard descriptor limit of 1024, from HELD_CREDENTIALS credentials. Part of #951.
The pages this layer wrote described the first cut of HLD-011, and the review of the layer found several of their claims false. They sized 192 for a descriptor limit of 1024 called the default soft limit on most Linux hosts and the smallest a container is commonly given (truth finding 2, house finding 4). A Go process never runs under that soft limit, since the runtime raises it to the hard one, so the pages now say the ceiling follows the hard limit read at startup, give the figures for 1024, 4096, 524288 and 1048576 and for Windows, and say what the ceiling costs where the limit is small (one credential can fill it in about fifteen seconds at the default rate) and where it is large (memory runs out first, at about 190 KiB a held call). They said without condition that a listen takes no slot while the code counted an older revision's (truth finding 4, house finding 12); now none is counted on any revision, and the pages say so and that each call of a batch counts on its own while a response to an elicitation never does. They said /health keeps answering under a limit of 1024 on the strength of one credential while a hundred reached 1141 to 1280 descriptors (truth finding 5, house finding 5); the claim now rests on a run from one credential and from a hundred under that limit, both at 394 descriptors with the refused connections closed, and the guide says the ceiling bounds calls and not idle connections. The security page said every call past the limit failed at its dial to GitLab, while 558 of the 1000 were served and 442 failed (house finding 6). And the sizing pages, in rewriting the sentence on limits that are not configurable, kept the failed-authentication budget among them, which --auth-failure-limit has configured since issue 790 (truth finding 10, house finding 9); the enterprise deployment guide said the same and is corrected with it. The HTTP server guide, security, resource consumption, troubleshooting, the enterprise and remote deployment guides, the cli, env and configuration references and the site pages in English and Spanish are brought to the derived ceiling, the two refusal channels and the busy words. Part of #951.
Widening a table re-adds every row of it, so the rows that carried an em dash before this layer widened three tables (the rejection codes in the HTTP guide and the troubleshooting tables of the guide and the site) now count as lines the branch adds, and make check-em-dash fails on them. Each dash becomes a colon, a pair of parentheses, or "none" in the headers column, and format_md_tables re-pads the tables. Part of #951.
… POST Mutating the held-call code showed two tests that hang instead of failing, and gremlins files a hang as TIMED OUT, which is no kill: the gate's condition with either && turned to ||, and the ceiling's sizing with its division turned to a multiplication (a ceiling of one), all three timed out. The gate test that sends POSTs it must not count waited on a channel the handler writes to; when a mutant refused the POST in the gate, no handler ran and the receive waited for ever. ServeHTTP has returned by then, so the receive is now immediate and a missing record is the failure. The whole-chain tests registered the stand-in GitLab's release before the MCP server's close, so the close ran first and waited for a held call that only the release could end; the release is now registered after the close and runs before it. Part of #951.
The paragraph opened with "two further limits are not configurable", named the two watcher ceilings, then called the configurable failed-authentication budget "a third" and added the held-call ceiling as "another". By its own count three limits were not configurable, and the sizing page counts the watcher ceilings as one limit and the held calls as the second. The two unconfigurable limits are now the watcher ceilings and the held calls, each in its own paragraph, followed by the configurable budget and its trap behind a proxy. The overlong line the earlier rewrite left inside the 80-column paragraph is rewrapped with it.
The transport budget remembers a (source, key) pair for one window and the limiter behind it counts a failure for one window, both inclusive of the last instant. Two boundary mutants of that edge lived in auth_gate.go, recorded by the cmd/server sweep as clock boundaries no test could schedule: charge re-opening a pair at exactly one window (`>` to `>=`), and the sweep dropping it at that instant. Either would charge the source a second time for a client the limiter still holds against it. testing/synctest can schedule that instant, since neither the budget nor the limiter touches a socket or starts a goroutine: a charge, a sleep of exactly one window, and a second charge (or a sweep and then a charge) must leave a two-client budget unblocked. Both mutants now die. The third boundary, window()'s `effectiveWindow <= 0`, stays recorded: newTransportBudget never stores a window of zero or less, so the guard is unobservable, and retiring that three-way fallback is issue 958's.
…package The branch had been measured over the files it writes only; the whole package still carried 24 lived, 67 not covered and 2 timed out mutants, all older than the held mode. This closes or classifies every one. Timed out: coldList's retry rule (series.go) had two mutants that ask again after an answer they should take. The tests that admit against a stand-in refusing tools/list then retried until callTimeout, five minutes per warm-up, so the suite timed out instead of failing on the test that pins the rule. The warm-up now has a ceiling of its own, warmUpTimeout, callTimeout in a run and fifteen seconds under the tests, where the stand-in answers at once; both mutants now fail the suite in under two minutes. Killed with a test: - waitUntil at an instant that is exactly now, on a cancelled run, asked on a fake clock (testing/synctest): waiting there raced a fired timer against the cancellation. - the latency stop's edge, moved into crossesLatencyCeiling so a tail exactly at the ceiling can be asked for; a measured tail cannot be. - admit runs its loop over the outcome slots, so the count of clients started and the count read back are one number; the two mutants of want-have were equivalent while they were two. - msLabel's band edges at one and ten milliseconds and cpuFigures on equal samples, two switch-case boundaries gremlins reported as not covered and no test pinned. Removed rather than recorded: the +1 on linearXTicks' first candidate (the spacing test already turns away the origin), the sampler's count of answered processes (only ever read for zero, now a bool), and the initial buffer of the goroutine-dump scanner (only the line ceiling matters). The rest are the recorded shapes of docs/development/testing/README.md, listed in the pull request.
Two tests failed on Windows. The validation cases for a held run that measures set a descriptor limit, which validate refuses on Windows because the shell that would set it is not there, so the host decided the verdict; the cases now pin the platform through the runtimeGOOS seam, and the refusal keeps the test of its own. The /health probe test demanded a positive elapsed time from a loopback answer, which can measure 0 where the monotonic clock advances in timer ticks; the handler now waits past a tick before it answers.
The held run reaches the server it started, over plain HTTP on the loopback interface, and wrote each of those addresses as a string beginning "http://", which SonarCloud reads as clear-text traffic (go:S5332) on every new line that does it. loopbackURL builds them from a url.URL and says in one comment why plain HTTP is the right scheme there, so the next mode of the benchmark reuses the reason instead of repeating the literal.
The held-call flood opens six hundred loopback connections at once to outrun a descriptor limit, and on windows-latest, which has none, only 444 of 600 were answered with 503 and the test could not say what the rest got. Most likely the burst overflowed the listen queue, which refuses a connection with WSAECONNREFUSED before the server accepts it. flood now keeps every connection in flight at once where the limit is what is measured and goes in waves of 64 on Windows, and a failure names each ending that was not a 503.
8e73aaa to
7cbe64d
Compare
|



A
tools/callholds itsPOSTfor as long as the call runs: up to an hour for a pipeline wait, and any call for as long as GitLab keeps it waiting. The rate limit bounds how often a credential calls, not how many of its calls are still running, and the pool bounds entries, not what an entry holds, so nothing bounded how many calls one credential, or all of them, could keep open (finding F-31). Under a hard descriptor limit of 1024 the process held about five hundred and then stopped accepting connections at all,/healthamong them.This is the second of issue 951's three layers. It adds register row
HLD-011: a ceiling on the calls the process holds open, keyed on the process and set by no flag. The stateful sessions, the other half of F-31, are the third layer.What it is sized from. The descriptor limit the process runs under, read once at startup: an eighth of it spare, one descriptor for each of the 512 listen streams (
HLD-002), and two for each held call (the caller's connection and the one to GitLab). The Go runtime raises the soft limit to the hard one beforemain, so the figure follows the hard limit: 192 under 1024, 1536 under 4096, 229120 under the 524288 a systemd service gets by default, 458496 under 1048576. Windows has no such limit to read and gets the figure for 1024. No flag moves it; raising the descriptor limit raises it together with what it protects. The startup line announces it asheld_requests_per_process.What it counts. Every call that reaches GitLab (the methods
MeterForcharges to the tool-call and completion buckets:tools/call,resources/read,resources/subscribe,prompts/get,completion/complete), asubscriptions/listenaside, counted where the SDK dispatches it. So each call of a JSON-RPC batch counts on its own, and a response a client sends to a request of the server's own (its answer to an elicitation), a notification, or a listen on any revision never does. On protocol 2026-07-28 or later, where the SDK holdsMcp-Methodto the body and refuses batches, the gate takes the slot itself after admission and hands it to the call through the carrier, so the refusal there happens before the SDK reads a byte.How it refuses. Always in the words
This server is busy. Retry later., naming no bound; the log line (request refused: too many requests held across the process) namesscope=processand the figure. On 2026-07-28 or later: a gate 503,-50300,Retry-After: 30, and the connection closed, so refused callers do not accumulate descriptors of the limit the ceiling protects. On older revisions: the way the rate limit refuses the same method (atools/callas a result withisError;resources/read,resources/subscribeandprompts/getwith-42900; a completion as an empty completion). Nothing is charged, and the credential's rate is not spent: the middleware runs before the rate limit.Measured
Truenas,
cmd/bench_resources -heldagainst a stand-in GitLab that holds every project read, headers first. Per held call, linear to 4000 calls: 2.00 descriptors, 6.00 goroutines, about 51 KiB of live heap and about 190 KiB resident./healthThe first cut of this layer reached 1141 to 1280 descriptors with a hundred credentials, because the refused connections stayed open; the refusal now closes its connection.
make bench-heldruns the ladder (HELD_CREDENTIALS=100for the second row).test/e2e/httpholds the same behaviour against the real binary started underulimit -n 1024: 192 calls held, a modern call refused in the gate with the connection closed, an older revision's call refused as a tool error, a batch of two refused call by call, a notification accepted, a listen acknowledged, 600 more calls offered at once all refused while/healthanswers, and every held call served once GitLab answers.What the review of the first cut found, and what changed
TestRegisterOAuthMCPHandlers_RefusesPastTheProcessHeldCeilingfails ifregisterOAuthMCPHandlersstops passing the count.TestHeldCall_TakesOneSlotAndLeavesItsEntryEvictablepins that a held call leaves its entry evictable and that the call outlives the eviction./healthassertion could not fail; the binary now runs underulimit -n 1024, where 792 held calls would not fit.cmd/bench_resourcesis now measured whole andcmd/server'smain.gois measured too; the results, and what closed each survivor, are under Gates.Decided on issue 951
The six questions this layer put are answered, each as proposed here, so nothing in this layer's row, pins, gate, e2e expectation or pages changes. The layer of this stack that records issue 951's decisions writes each answer and its reason where the tree still presented the question: the register row's comment, the specification, ADR-0023, the HTTP server, enterprise, remote deployment, security and resource consumption pages, and the site in English and Spanish.
FallbackDescriptorLimit1024) rather than no ceiling: 192 protects no descriptor there, and it is kept so the platform is not left unbounded by default.HLD-001stands besideHLD-002. The credential is a key a caller can mint (INV-003), so a per-credential number multiplies with every token a caller mints and bounds nothing the process ceiling does not. One credential can therefore fill the ceiling where the limit is small, in about fifteen seconds at the default rate (a burst of 40, then ten a second), and the operator pages say so.GOMEMLIMIT; the pages set the container's as the bound and describeGOMEMLIMITas a soft target the runtime collects toward and live held calls pass, so it is a lever and not that bound.POL-006's probe and, at the pool's bound, another credential's quiet entry evicted,POL-002), in oauth mode oneADM-014slot, and none of its rate.HLD-011falls under that exception rather than needing its own.The two confirmations issue 951 asked for: a held call does not make its pool entry busy (
POL-003unchanged, and now pinned by a test); the process listing bucket followsRTC-003(the first layer, 951a).Decisions I took here
UpstreamRetryAfter(30 s, fixed) andCodeUnavailable(-50300) rather than adding values, and says only that the server is busy; the log line alone names the scope and the figure (decided on issue 951, item 5).POL-003).cmd/bench_resourcesto remove equivalent mutants rather than record them (admitnow runs over one count, the sampler's count of answered processes is a bool, the goroutine-dump scanner has no initial buffer, the first chart tick candidate lost its+1), moved the latency stop's edge intocrossesLatencyCeilingso a test can reach it, and gave the warm-up its own ceiling (warmUpTimeout); each change preserves behaviour for every input that reaches it, and the package stays at 100% of statements and 1322 of 1322 conditions.auth_gate.go:793(effectiveWindow <= 0) stays classified rather than deleted: retiring that three-way window fallback is issue 958's, asAUB-002and Close the mutation and condition sweep of cmd/server #1015 already record.Not done
cmd/serverwas mutated over the four files this layer writes, not whole;main.go's survivors are the ones Close the mutation and condition sweep of cmd/server #1015 recorded on identical code.Validation checklist (tenant-policy-spec.md)
KeyProcess, which no caller can mint. No new key.-50300, fixed 30 sRetry-After, no challenge, connection closed); in-band on older revisions (tools/callas a tool error;resources/read,resources/subscribeandprompts/getas RPC-42900;completion/completeas an empty completion). Retry-later class throughout, textThis server is busy. Retry later.. On stdio: none.Source: Derived, fromRLIMIT_NOFILEread at startup and three pinned constants (HeldRequestDescriptors2,DescriptorSpareDivisor8,FallbackDescriptorLimit1024) besideListenStreamsPerProcess; a floor of one call; zero is not applicable; no flag or variable, so no malformed-value policy.StdioKeynone). The middleware counts only calls that arrived on an HTTP POST, so a stdio server carries it and counts nothing.make bench-fairnessanswers a claim this ceiling does not make.Gates
Measured on this layer before it was rebased onto main at 5da9699: the line numbers below are that tree's, and the one commit cited below (fef2e45) is named as it sits in this stack; main's server card change (#1061) moved
main.go's lines and none of the code these figures describe. The stack was then rebased onto 50ef2cf, whose only change to a file this layer writes is the serialtypecheck blank import incmd/audit_tenancy/main_test.go, and at its place in this stack on that base the layer built and vetted, the e2e gitlab, http and stdio packages compiled, the unit suites ofcmd/server,internal/tenancy,internal/oauthandcmd/audit_tenancypassed, andmake check-tenancyread the same 83 rows and 299 sites. The stack now sits on b44ab96, which adds #1083, #1084 and #1085; they change the upstream register, indirect module versions and the site's package pins, none of them a file this stack writes, and every commit of the stack applied unchanged.go test -shuffle=on: 100.0% of statements incmd/server,internal/tenancy,cmd/bench_resourcesandcmd/audit_tenancy;-raceon the held tests; the wholetest/e2e/httpmodule (578 tests, the Docker-only proxy and balancer cases skipped); the e2e gitlab and stdio packages compile.e2eandhttpe2etags: 0 issues. markdownlint andformat_md_tables --check: clean.make check-tenancy: 83 rows, 21 failures and 299 declared sites over 202 packages (21 refusal returns, 80 refusals, 13 reasons and 31 settings read); 0 findings, 56 declarations exempted.check-dead-consts,check-test-file-names,check-test-subtests,check-test-goroutines,check-doc-tool-namesandcheck-action-idspass.internal/tenancy408/408,cmd/bench_resources1322/1322,cmd/audit_tenancy1002/1002.cmd/serveris measurable through a staged copy without its non-Linux twins (the issue 1017 approach): 2040/2082 (2039 on one of three runs, a timing-dependent condition outside these files), with none of the open conditions inheld.go,descriptor_limit.goor the lines this layer changed inauth_gate.goandmain.go; all of them predate this change,auth_gate.go:793among them (the unobservable guard below).--workers 2 --timeout-coefficient 20,INVERT_LOGICALon):internal/tenancy, whole package: 226 killed, 0 lived, 0 timed out, 66 not covered, every one of them arithmetic in a constant declaration (codes.go26,values.go33,decision.go7), which has no statement for coverage to count; each value is pinned byTestValues_HoldTheirPins, and the count is the base's.cmd/server, scoped to the four files this layer writes (the house rule allows it for this package), in two runs:held.go,descriptor_limit.goandauth_gate.go: 99 killed, 1 lived, 8 not covered, 0 timed out (held.goanddescriptor_limit.goalone: 23 killed and nothing else). The first pass over the held files timed out three mutants, because two tests hung instead of failing when the gate refused a POST they expected to pass; both now fail at once (fef2e45). The transport budget's two window-edge boundaries (chargere-opening a pair at exactly one window, and the sweep dropping it then) are now killed: two tests land a charge exactly one window later on a fake clock (testing/synctest), which the cmd/server sweep (Close the mutation and condition sweep of cmd/server #1015) had recorded as a clock no test could schedule. The one lived iswindow()'seffectiveWindow <= 0(auth_gate.go:793):newTransportBudgetnever stores a window of zero or less, so the guard is unobservable, and Close the mutation and condition sweep of cmd/server #1015 recorded it and left it for issue 958, which retires that three-way fallback; deleting it here would be half of 958. The 8 not covered are constant initializers: the two JSON-RPC code literals, whose sign flipsTestRefusalCodes_MirrorTheirStatusAndSitWhereTheSpecificationAllowskills when they are applied by hand, and+to-between string constants, which does not compile.main.go: 338 killed, 4 lived, 25 not covered, 0 timed out. This layer's change to it carries no mutable token (a middleware registration, two struct fields, a log attribute and a help line). The 4 lived are the four boundaries Close the mutation and condition sweep of cmd/server #1015 recorded, on identical code (at lines 1899, 2605, 3244 and 3744 in Close the mutation and condition sweep of cmd/server #1015's tree, and 1911, 2629, 3273 and 3776 in the one measured here); the 25 not covered are 15 constant and variable initializers, 9switchcase expressions and the equivalent802:19Close the mutation and condition sweep of cmd/server #1015 recorded, which is where Close the mutation and condition sweep of cmd/server #1015 left the file.cmd/bench_resources, whole package, with the script's default--exclude-files=/so the out-of-process stand-in undertestdata/standinis not counted. Before this pass: 1173 killed, 24 lived, 67 not covered, 2 timed out, all in files this layer does not otherwise touch. After it: 1174 killed, 17 lived, 67 not covered, 0 timed out.coldList's retry rule mutated to ask again after an answer it should take: every test that admits against a stand-in refusingtools/listthen retried untilcallTimeout, five minutes a warm-up, and the suite timed out instead of failing the test that pins the rule. The warm-up now has its own ceiling (warmUpTimeout,callTimeoutin a run and fifteen seconds under the tests); both mutants are killed.waitUntilat an instant that is exactly now on a cancelled run (a fake clock); the latency stop's edge, moved intocrossesLatencyCeilingso a tail exactly at the ceiling can be asked for;admit'swant-have(twice), now the one count the loop runs over; and twoswitchcase edges gremlins reports as not covered that no test pinned,msLabelat 1 and 10 ms andcpuFigureson equal samples.+1onlinearXTicks' first candidate (the spacing test already turns the origin away), the sampler's count of answered processes (read only for zero, now a bool) and the initial buffer of the goroutine-dump scanner (only the line ceiling matters).docs/development/testing/README.mdrecords. Two sides that agree at the boundary:chart.go:276(Log10(0)is-Inf, which clamps to the floor wherev = lolands),chart.go:330andproc_sample.go:271(the assignment writes the value already held),chart.go:570(a candidate atmaxXis turned away by the spacing test),chart.go:718and:721(a shift by zero),fairness.go:242(at share zero the division is+Inffor every bound, all of which meter a positive rate, andmincaps it at the same default),fairness.go:617(zero times anything),fairness_result.go:550(a floor of zero is met by any busy figure, never negative),figures.go:402andrender.go:230(a string of exactly the slice's length is returned whole either way),proc_sample.go:32(a probe whose deadline has passed falls back to 100, whatgetconfanswers on Linux). Unreachable by construction:chart.go:299(ticks are whole steps and the bound is half a step past the last),fairness_result.go:715(reached only once the claim exceeds a spread of zero or more),fairness_result.go:734twice (a zero delta returns before either comparison, the tie-break shape),theme.go:96(the index is taken over text that begins with the selector, so it is never 0).switchcase expressions, which carry no statement counter, and 2 atfairness_run.go:277, a call argument after a function literal that the profile attributes to no block. Applied by hand, thefairness_run.go:277mutant is killed byTestDrive_SpreadsEachPopulationAcrossOnePeriodand the two protocol codes' sign flips byTestClassifyOutcome_KeepsRefusalsApartFromSuccessAndFailure; everyswitchcase boundary has a test at its edge now (the two that had none are above);+to-between string constants does not compile. gobco holds every case condition both ways (1322/1322).Part of #951.