Repository navigation
Measure django-tasks-rq and dramatiq in the queue benchmark - #89
Merged
Merged
Conversation
django-tasks-redis leaves the comparison for django-tasks-rq, the RQ backend of the django-tasks family, and dramatiq joins it. Both are measured on the same trivial echo task and the same queue as the rest: - django-tasks-rq drains with `rqworker --burst` and its own Job class. RQ's worker forks a work horse per job, so its 80 tasks/s includes that fork, and its enqueue path costs 793 tasks/s against threadmill's 7,969. - dramatiq and celery read four messages per worker thread, celery's shipped prefetch multiplier. Dramatiq's Redis consumer polls rather than blocks and sleeps a jittered backoff once its window fills, so 669 tasks/s still measures that backoff more than the queue. The same worker drains 451 tasks/s at two messages, 1,983 at 16 and 5,658 at 64, where the backoff stops setting the rate; its own default is two per worker thread. - threadmill, django-tasks-db and django-tasks-rq read one task at a time, because their workers block on an empty queue and gain nothing from a window. The status check compares by value, since django-tasks-rq returns the django-tasks backport's TaskResultStatus, a different enum with the same value. The chart grows with its row count, and the README alt text carries the new numbers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The queue comparison still measures django-tasks-redis, which is not the RQ backend the Django ecosystem lists, and it has no dramatiq entry.
rqworker --burst --job-class django_tasks_rq.Jobwith itsRQ_QUEUESmapping.TaskResultStatus, a different enum with the same value.