Data Quality/Cleanup: Querying Save Blocked Records

[bronwyncombs]

Records with a save block in Specify have one or more fields with a quality issue. No other modifications can be made to the record until the save-blocker issue is resolved.

Save can be blocked due to:

• Required field without a value (e.g., Locality Name is missing)
• Field with a value that doesn't conform to an assigned field format (e.g., Accession Number is 000045 but format requires A-000045)
• Field with an otherwise unexpected value (e.g., coordinates with untranslated direction characters)

Currently, records with a save block can be identified only by seeing that the Save button is red when viewing the full record. This means that there is no straightforward way to know how many records have a quality issues or what the most prevalent quality issue is. The capacity to resolve any data quality issues across records in Specify is also very limited since there is no way to efficiently find and effectively group save block records.

Solution:

Enable returning all save block records in query builder. Then, all save-blocking data issues can be easily found and corrected. Data cleanup can be tracked in statistics if desired.

This might be done by adding an isSaveBlocked field for each table level so users can add isSaveBlocked=true condition. I could also see it achieved through its own table (similar to spAuditLog, i.e., SpDataQuality) that can be queried on to return all save block records and get filtered by table from there.