feat(kit): add hosted capacity guardrails - #269
Merged
Merged
Conversation
Document the hosted service as shared, community-funded best-effort infrastructure and guide high-volume teams toward sponsorship or self-hosting. Add key, trusted-source-IP, and process concurrency guards for verification alongside Fly proxy limits, with SERVICE_BUSY contracts and regression coverage.
hyochan
added
🎯 feature
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (12)
🚧 Files skipped from review as they are similar to previous changes (12)
📝 WalkthroughWalkthroughThe PR adds scoped verification concurrency limits with bounded ChangesHosted Capacity Controls
Estimated code review effort: 4 (Complex) | ~50 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant PurchaseVerificationRoute
participant inFlightLimitMiddleware
participant VerificationHandler
Client->>PurchaseVerificationRoute: Send verification request
PurchaseVerificationRoute->>inFlightLimitMiddleware: Apply scoped concurrency guard
alt Capacity available
inFlightLimitMiddleware->>VerificationHandler: Admit request, increment counters
VerificationHandler-->>inFlightLimitMiddleware: Result or error
inFlightLimitMiddleware->>inFlightLimitMiddleware: Decrement counters in finally
inFlightLimitMiddleware-->>Client: Response with concurrency headers
else Capacity exhausted
inFlightLimitMiddleware-->>Client: 503 SERVICE_BUSY with scope and Retry-After
end
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint install failed. For unrecoverable errors, disable the tool in CodeRabbit configuration. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Member
Author
Previewpr-kit-capacity-guardrails.mp4 |
There was a problem hiding this comment.
Pull request overview
This PR adds explicit hosted-capacity/fair-use messaging across Kit + Docs and introduces server-side concurrency guardrails for /api/v1/purchase/verify, returning 503 SERVICE_BUSY with retry guidance when in-flight verification capacity is saturated.
Changes:
- Added per-key / per-source-IP / per-process in-flight verification caps and surfaced new
X-Concurrency-*headers in responses and OpenAPI. - Updated Kit + Docs content to set expectations about shared best-effort hosted capacity, funding options (GitHub Sponsors/OpenCollective), and self-hosting guidance.
- Added Fly Proxy HTTP request concurrency limits and expanded operational/security/ToS documentation to reflect defense-in-depth safeguards.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| packages/kit/src/pages/landing.tsx | Updates landing copy + CTAs to emphasize fair-use hosted service and sponsorship/self-host options. |
| packages/kit/src/pages/docs/sections/operations.tsx | Documents hosted fair use + new concurrency capacity limits and tuning env vars. |
| packages/kit/src/pages/docs/sections/api.tsx | Updates client refresh guidance and documents 503 SERVICE_BUSY + concurrency headers. |
| packages/kit/src/pages/blog/iapkit-joins-openiap.tsx | Adjusts announcement FAQ/copy to reflect fair-use safeguards and high-volume coordination. |
| packages/kit/src/pages/auth/organization/usage.tsx | Updates usage page messaging + adds GitHub/OpenCollective/self-host links and capacity-planning contact. |
| packages/kit/src/content/terms-of-service.md | Adds fair-use/capacity/abuse section; updates prohibited actions and renumbers sections. |
| packages/kit/src/components/FreeTransitionNotice.tsx | Updates subscription transition notice to reference fair-use safeguards. |
| packages/kit/server/api/v1/routes.ts | Wires inFlightLimitMiddleware, adds OpenAPI 503 + concurrency headers for verify route. |
| packages/kit/server/api/v1/routes.test.ts | Extends OpenAPI/spec tests and asserts concurrency headers on successful verification. |
| packages/kit/server/api/v1/rate-limit.ts | Updates comments to reflect hosted fair-use framing (no functional changes in shown diff). |
| packages/kit/server/api/v1/in-flight-limit.ts | Introduces new middleware implementing in-flight concurrency caps with 503 SERVICE_BUSY. |
| packages/kit/server/api/v1/in-flight-limit.test.ts | Adds tests verifying caps, anti-key-rotation behavior per IP, and slot release on errors. |
| packages/kit/SECURITY.md | Clarifies DoS scope expectations and lists defense-in-depth controls (rate/replay/size/concurrency). |
| packages/kit/README.md | Adds hosted fair-use/capacity-planning guidance and documents concurrency safeguards + env vars. |
| packages/kit/public/llms.txt | Updates LLM quick reference to include fair-use, 503 behavior, and concurrency header info. |
| packages/kit/public/llms-full.txt | Updates LLM full reference with 503 + concurrency and hosted-capacity guidance. |
| packages/kit/fly.toml | Adds Fly HTTP service request concurrency soft/hard caps per machine. |
| packages/kit/COST-SAFETY.md | Updates cost-safety narrative to include new concurrency controls and hosted-capacity framing. |
| packages/kit/convex/plans.ts | Clarifies that monthly accounting is non-blocking and edge safeguards may throttle. |
| packages/kit/.env.example | Documents new VERIFY_* env vars and retry-after config. |
| packages/docs/src/pages/sponsors.tsx | Adds hosted IAPKit sustainability/fair-use section + OpenCollective option and CTA updates. |
| packages/docs/src/pages/docs/kit-backend.tsx | Adds a hosted-capacity section with guidance, limits, and sponsorship/self-host recommendations. |
| packages/docs/public/llms.txt | Updates generated timestamp. |
| packages/docs/public/llms-full.txt | Updates generated timestamp and includes hosted-kit fair-use/concurrency guidance. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/kit/COST-SAFETY.md`:
- Line 3: Update the effective date in the COST-SAFETY document header to August
1, 2026, unless August 2, 2026 is intentionally scheduled; in that case,
explicitly label it as the future effective date.
- Around line 66-69: Update the concurrency behavior description in
COST-SAFETY.md to say requests are rejected when an axis is full or reaches its
configured limit, rather than whenever any axis is occupied; preserve the
existing 503, headers, slot-release, and idle-entry cleanup details.
In `@packages/kit/public/llms.txt`:
- Around line 124-125: Update the 429 RATE_LIMITED descriptions to document
API-key, source-IP, and process limiter buckets, so clients can use
X-Concurrency-Scope to identify the rejection; in packages/kit/public/llms.txt
lines 124-125, packages/kit/public/llms-full.txt lines 326-340, and
packages/docs/public/llms-full.txt lines 2152-2160, revise the earlier entries
with these scopes. In packages/kit/src/pages/docs/sections/api.tsx lines
603-611, update the existing 429 row to distinguish key, source-IP, process, and
replay-guard rejections.
In `@packages/kit/server/api/v1/in-flight-limit.ts`:
- Around line 89-102: Update the apiKeyHash misconfiguration branch in the
concurrency guard to return the same X-Correlation-Id and X-Concurrency-Limit,
X-Concurrency-Remaining, and X-Concurrency-Scope headers as the normal rejection
path, preserving the documented 500 response contract.
In `@packages/kit/server/api/v1/routes.test.ts`:
- Around line 73-75: Update the /purchase/verify test around the
X-Concurrency-Remaining assertion to avoid depending on sharedVerifyState call
order. Inject isolated state for the route under test, or derive the expected
remaining value from the relevant key’s current state, while preserving the
existing limit and scope assertions.
In `@packages/kit/src/pages/docs/sections/api.tsx`:
- Around line 517-522: Update the concurrency-header documentation to state that
requests must reach the in-flight guard, rather than merely pass body
validation, before receiving X-Concurrency-* headers. Apply this wording in
packages/kit/src/pages/docs/sections/api.tsx lines 517-522,
packages/kit/public/llms.txt lines 127-134, packages/kit/public/llms-full.txt
lines 318-324, and packages/docs/public/llms-full.txt lines 2148-2150.
- Around line 499-502: Update the response-header guarantee in the API
documentation paragraph near the authenticated response section so it does not
claim correlation and rate-limit headers on every authenticated response. Either
document route-level coverage for paths such as subscriptions and products admin
routes, or narrow the sentence to only the headers guaranteed by each documented
middleware chain.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 35c1ea40-b849-45e0-9a66-26115fd0a63c
📒 Files selected for processing (24)
packages/docs/public/llms-full.txtpackages/docs/public/llms.txtpackages/docs/src/pages/docs/kit-backend.tsxpackages/docs/src/pages/sponsors.tsxpackages/kit/.env.examplepackages/kit/COST-SAFETY.mdpackages/kit/README.mdpackages/kit/SECURITY.mdpackages/kit/convex/plans.tspackages/kit/fly.tomlpackages/kit/public/llms-full.txtpackages/kit/public/llms.txtpackages/kit/server/api/v1/in-flight-limit.test.tspackages/kit/server/api/v1/in-flight-limit.tspackages/kit/server/api/v1/rate-limit.tspackages/kit/server/api/v1/routes.test.tspackages/kit/server/api/v1/routes.tspackages/kit/src/components/FreeTransitionNotice.tsxpackages/kit/src/content/terms-of-service.mdpackages/kit/src/pages/auth/organization/usage.tsxpackages/kit/src/pages/blog/iapkit-joins-openiap.tsxpackages/kit/src/pages/docs/sections/api.tsxpackages/kit/src/pages/docs/sections/operations.tsxpackages/kit/src/pages/landing.tsx
The verification source axis trusts only Fly's `fly-client-ip`, so a deployment that does not run behind Fly resolves every caller to one shared "unknown" source. That makes VERIFY_MAX_IN_FLIGHT_PER_IP a second process-wide cap below VERIFY_MAX_IN_FLIGHT, which self-hosters tuning these values would otherwise hit without explanation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
10 tasks done
hyochan
added a commit
that referenced
this pull request
Aug 4, 2026
The apple version badge pinned a filter=2.* tag glob, so it kept rendering v2.4.4 forever after the 3.x releases while the CocoaPods badge beside it showed v3.0.1. It now reads $.apple from openiap-versions.json with the same dynamic-json pattern the spec badge already uses, so a future major bump can never strand it again. Verified against the live shields endpoint. Also aligns the kit line with the fair-use wording adopted in #269 and lists packages/mcp-server, which the "all OpenIAP packages" section omitted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Safeguards
503 SERVICE_BUSYwithRetry-Afterand concurrency scope headers instead of queueingVerification
audit:docs,audit:release-state, andaudit:parityflyctl config validatecould not be run locally because the CLI has no auth token;fly.tomlparses locally and CI remains the remote validation gate.Summary by CodeRabbit
New Features
503 SERVICE_BUSYresponses with retry guidance and capacity headers.Documentation
Bug Fixes