feat(transform-eval): per-org concurrency bound for vector subprocess (SC-2)#499
Merged
Merged
Conversation
Add src/lib/org-concurrency.ts: an in-process per-(org,key) async semaphore (withOrgConcurrencyLimit) that caps concurrent fn executions, queues the rest FIFO, and always releases on success/throw. A missing/empty orgId falls back to a shared "global" bucket; idle buckets are pruned. Thread an optional orgId into evaluateVrl(): when provided, the vector subprocess spawn runs under withOrgConcurrencyLimit(orgId, "vector-eval", 4) so one tenant's concurrent VRL evals (live-tap, cost what-if, unit-test "run all", AI propose auto-fix loop) can no longer spawn unbounded vector processes and starve the shared host. The no-op fast path never takes a slot, and behavior is identical when orgId is omitted (backward-compatible). Pass ctx/input organizationId at the org-scoped call sites: vrl runUnitTests/runPipelineUnitTests/testAgainstCapture, tap-capture testTransform, cost-model simulateTransform, and proposed-change VRL validation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SC-2 — per-org fairness for the heavy
vectorsubprocessevaluateVrl()(src/server/services/transform-eval.ts) spawns avectorsubprocess (execFile, 15s) on every VRL eval. There was no cross-request per-org bound, so one tenant issuing many concurrent evals (live-tap iteration, cost what-if, unit-test "run all", the AI propose auto-fix loop) could spawn unboundedvectorprocesses and starve the shared host.Change
src/lib/org-concurrency.ts—withOrgConcurrencyLimit(orgId, key, max, fn): an in-process per-(org, key) async semaphore. Caps concurrentfnexecutions for the pair, queues the rest FIFO, hands a freed slot directly to the next waiter, and always releases on success/throw. A missing/emptyorgIdcollapses to a shared"global"bucket; idle buckets are pruned so the Map tracks live concurrency, not historical org count. Single-process (per Node worker) — fairness within a host, not a distributed limiter.evaluateVrl(source, events, opts?)gains optionalopts.orgId. When set, the subprocess spawn runs underwithOrgConcurrencyLimit(orgId, "vector-eval", 4). The no-op fast path (empty program/input) never takes a slot, and behavior is identical whenorgIdis omitted (backward-compatible).{ orgId: ctx/input.organizationId }:vrl.ts—runUnitTests,runPipelineUnitTests(viarunTestsAgainstSource),testAgainstCapturetap-capture.ts—testTransform(live-tap)cost-recommendation-procedures.ts—simulateTransform(cost what-if)proposed-change.ts— VRL validate + bounded auto-fix loopTests
src/lib/__tests__/org-concurrency.test.tsuses controllable deferred barriers (no real timers): caps concurrency (max=2, 5 tasks → peak 2), drains FIFO in order, isolates different orgs and different keys, falls back empty/whitespace orgId → shared"global"bucket, and releases the slot on throw.vitest run src/lib/__tests__/org-concurrency.test.ts src/server/services/__tests__/transform-eval.test.ts→ 15 passedtap-capture,vrl-unit-test,vrl-pipeline-unit-test,cost-recommendation-simulate,proposed-change) → 60 passed (updated theevaluateVrlarg assertions to the new 3-arg form)tsc --noEmitfiltered to changed files: clean — only the known worktree-local@clickhouse/clientmodule-resolution error remains (unrelated to this change).Notes