Skip to content

perf(processor): run a one-thread task on the calling thread - #1297

Open
zahariash wants to merge 1 commit into
LadybugDB:mainfrom
zahariash:perf/inline-one-thread-tasks
Open

zahariash wants to merge 1 commit into
LadybugDB:mainfrom
zahariash:perf/inline-one-thread-tasks

Conversation

@zahariash

Copy link
Copy Markdown
Contributor

This work was produced with the help of language models.

Fixes #1296. Part of #1290.

scheduleTaskAndWaitOrError pushed every task to the queue and woke the whole worker pool, even when only one thread could register. A one-row READ_FTABLE source (RETURN 1 and other constant queries) was also marked parallel, so every worker cloned the pipeline before finding no morsel: 0.94 ms to execute RETURN 1 on a fresh database, 157 us per query warm.

Fix. A task limited to one thread runs on the calling thread and the pool stays asleep. Workers and inline callers share the database's numWorkerThreads slots, so max_num_threads still bounds the threads running queries across connections: 8 connections each running a heavy single-threaded query on a 2-thread database use 2.73 cores, as on main. A caller that finds no free slot queues as before; a scope guard returns the slot, also when the task throws, and wakes one worker while tasks are queued. Queries with a timeout keep the queue path, since the waiting caller enforces the timeout, as do callers that launch their own worker thread (GDS, the LLM extension); that thread reuses the caller's slot for nested one-thread tasks. After a task's dependencies finish, a READ_FTABLE scan that fits in one morsel is lowered to one thread. No extension ABI change: members appended, existing overloads kept.

Performance. RETURN 1 in a fresh in-memory database, query-summary execute time, median of 8 processes; warm is a C API loop of 20000 queries.

main this PR
1st query 0.94 ms 0.54 ms
2nd query 0.30 ms 0.04 ms
3rd query 0.20 ms 0.02 ms
warm 157 us 59 us

Nine LDBC SF1 queries at one thread (joins, aggregates, sort, a 100K-row collect): identical rows, instructions within ±0.2%.

Tests. Four TaskSchedulerTest cases: a one-thread task runs on the calling thread; 8 callers on a 2-thread database peak at 2 threads (8 without the slot accounting); tasks queued while every slot is held inline run side by side once the callers return (fails in 20 of 20 runs if a slot release wakes a worker only when the pool was full); a nested one-thread task under a 1-thread limit reuses the caller's slot (hangs otherwise). Full ctest on this branch: 3104/3104. One earlier run had a single failed reader query in ApiTest.ConcurrentCheckpointRead1160; it did not come back in the re-run or in 450 stress runs of that test, and 450 with main's scheduler did not show it either.

scheduleTaskAndWaitOrError pushed every task to the queue and woke the
whole worker pool, even when only one thread could register. A one-row
READ_FTABLE (RETURN 1 and other constant queries) was also marked
parallel, so every worker cloned the pipeline before finding no morsel.

A task limited to one thread now runs on the caller. Workers and inline
callers share numWorkerThreads slots under the scheduler mutex, so
max_num_threads still bounds the threads running queries across
connections; a caller that finds no free slot uses the queue, and a
thread that already holds a slot reuses it. Queries with a timeout and
callers that pass launchNewWorkerThread keep the queue path. After a
task's children finish, a READ_FTABLE source that fits in one morsel is
lowered to one thread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zahariash

Copy link
Copy Markdown
Contributor Author

Running a one-thread task on the calling thread changes one Python driver test: test_close_with_query_in_flight[close_from_a_udf] fails, 3 of 3 runs on this branch and none on main. The UDF in that test runs in a one-thread task, so it now runs on the thread that issued the query, and ladybug-python's close() sees that connection's own call on that thread and refuses with "close() cannot be called from a call on the same connection" instead of the UDF message. The close is still refused, and the CI jobs here don't run the driver suite. The same mismatch already happens on main for a UDF folded while binding; LadybugDB/ladybug-python#71 fixes both, and with it the driver suite against this branch (without the torch-geometric and JSON-extension tests) matches main.

I also stress-tested this PR together with #1286, #1300, #1294 and a local parser change: full ctest 3109/3109, our workload simulator (300 workloads), 8 threads of checked queries with random timeouts, interrupts and a writer at 1, 4 and 8 engine threads (about 381k queries, no errors or hangs), and LDBC SF1, where all 44 queries return the same results as main.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A one-thread task wakes the whole worker pool, 0.9 ms executing a constant query

1 participant