Skip to content

Saving a large loan re-reads django_migrations once per saved row (uncached businessrules migration check) #8530

Description

@urban011

Specify 7.12.1.1, docker. Saving a loan with 755 loan preparations takes 60 to 90 seconds, and py-spy dumps of the gunicorn worker taken during two of those saves both sit in the same two functions:

_initial_businessrules_migration_applied (specifyweb/backend/businessrules/uniqueness_rules.py:98)
_cached_businessrules_migration_applied (specifyweb/backend/businessrules/uniqueness_rules.py:113)
validate_unique (specifyweb/backend/businessrules/uniqueness_rules.py:204)
handler (specifyweb/backend/businessrules/orm_signal_handler.py:50)
custom_save (specifyweb/specify/models.py:22)
save_spauditlog (specifyweb/specify/models.py:6096)
_log (specifyweb/backend/workbench/upload/auditlog.py:127)
log_action (specifyweb/backend/workbench/upload/auditlog.py:74)
update (specifyweb/backend/workbench/upload/auditlog.py:71)
view (specifyweb/specify/views.py:68)

_cached_businessrules_migration_applied only uses _uniqueness_migration_cache when the cache has been activated through cache_uniqueness_rules(), and nothing on the API save path activates it, so every call runs MigrationRecorder(connections["default"]).applied_migrations(), which reads all of django_migrations (107 rows on our database). validate_unique runs from the pre-save signal for every saved row, the loan preparations themselves and each spauditlog row alike (the other dump has the same top frames with custom_save called straight from view instead of through the audit log).

The numbers from the one that stalled: PUT /api/specify/loan/60/ on a loan with 755 loan preparations. The audit log shows each of the three saves the curator made in 15 minutes wrote 755 LoanPreparation and 754 LoanReturnPreparation audit rows, so a single click is roughly 3,000 row saves and roughly 3,000 full reads of the migrations table, which is where the minute goes.

The comment in that function already notes a migration wouldn't be reversed while Specify is running, so either activating cache_uniqueness_rules() around the API save and delete views, or caching a True result for the life of the process, should bring this down to one query per request. Happy to test a patch on our instance, we have loans this size in regular use.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions