Specify 7.12.1.1, docker. Saving a loan with 755 loan preparations takes 60 to 90 seconds, and py-spy dumps of the gunicorn worker taken during two of those saves both sit in the same two functions:
_initial_businessrules_migration_applied (specifyweb/backend/businessrules/uniqueness_rules.py:98)
_cached_businessrules_migration_applied (specifyweb/backend/businessrules/uniqueness_rules.py:113)
validate_unique (specifyweb/backend/businessrules/uniqueness_rules.py:204)
handler (specifyweb/backend/businessrules/orm_signal_handler.py:50)
custom_save (specifyweb/specify/models.py:22)
save_spauditlog (specifyweb/specify/models.py:6096)
_log (specifyweb/backend/workbench/upload/auditlog.py:127)
log_action (specifyweb/backend/workbench/upload/auditlog.py:74)
update (specifyweb/backend/workbench/upload/auditlog.py:71)
view (specifyweb/specify/views.py:68)
_cached_businessrules_migration_applied only uses _uniqueness_migration_cache when the cache has been activated through cache_uniqueness_rules(), and nothing on the API save path activates it, so every call runs MigrationRecorder(connections["default"]).applied_migrations(), which reads all of django_migrations (107 rows on our database). validate_unique runs from the pre-save signal for every saved row, the loan preparations themselves and each spauditlog row alike (the other dump has the same top frames with custom_save called straight from view instead of through the audit log).
The numbers from the one that stalled: PUT /api/specify/loan/60/ on a loan with 755 loan preparations. The audit log shows each of the three saves the curator made in 15 minutes wrote 755 LoanPreparation and 754 LoanReturnPreparation audit rows, so a single click is roughly 3,000 row saves and roughly 3,000 full reads of the migrations table, which is where the minute goes.
The comment in that function already notes a migration wouldn't be reversed while Specify is running, so either activating cache_uniqueness_rules() around the API save and delete views, or caching a True result for the life of the process, should bring this down to one query per request. Happy to test a patch on our instance, we have loans this size in regular use.
Specify 7.12.1.1, docker. Saving a loan with 755 loan preparations takes 60 to 90 seconds, and py-spy dumps of the gunicorn worker taken during two of those saves both sit in the same two functions:
_cached_businessrules_migration_appliedonly uses_uniqueness_migration_cachewhen the cache has been activated throughcache_uniqueness_rules(), and nothing on the API save path activates it, so every call runsMigrationRecorder(connections["default"]).applied_migrations(), which reads all ofdjango_migrations(107 rows on our database).validate_uniqueruns from the pre-save signal for every saved row, the loan preparations themselves and eachspauditlogrow alike (the other dump has the same top frames withcustom_savecalled straight fromviewinstead of through the audit log).The numbers from the one that stalled: PUT /api/specify/loan/60/ on a loan with 755 loan preparations. The audit log shows each of the three saves the curator made in 15 minutes wrote 755 LoanPreparation and 754 LoanReturnPreparation audit rows, so a single click is roughly 3,000 row saves and roughly 3,000 full reads of the migrations table, which is where the minute goes.
The comment in that function already notes a migration wouldn't be reversed while Specify is running, so either activating
cache_uniqueness_rules()around the API save and delete views, or caching a True result for the life of the process, should bring this down to one query per request. Happy to test a patch on our instance, we have loans this size in regular use.