fix(queries): skip missing tree ranks

[grantfitzsimmons]

Fixes #3351

If a query, data export, (or soon, data view query) is executed in a collection that doesn't have matching ranks, this prevents it from crashing. This is a common problem for databases that share queries between collections, especially when those collections exist in other disciplines where some tree ranks might not exist. It also allows us to ship default queries or export mappings that function even if the ranks included in the defaults are not present in the collection(s).

Checklist

• Self-review the PR after opening it to make sure the changes look good and
  self-explanatory (or properly documented)
• Add relevant issue to release milestone
• Add pr to documentation list
• Add automated tests

Testing instructions

1. Open or create a stored query containing a tree-rank field that is not present in the active collection. (If you are having a hard time finding one, I guess you can use this: FakeRankQuery.json)
2. Execute the query in the table view and confirm that it does not crash and that the remaining result columns stay aligned with their headers.
3. Apply a filter or sort using the unavailable rank and confirm that the query still executes while ignoring that field.
4. Export the same query and confirm that the export completes and the unavailable field is omitted or empty without shifting other columns.

There is automated coverage should verify both a missing displayed rank (preserving the result-column position) and a missing filtered/sorted rank (ignoring the field without changing the result set).

Summary by CodeRabbit

• Bug Fixes
  • Fixed query results so unresolved fields retain the correct column positions.
  • Prevented invalid filtering and sorting when a query field cannot be resolved.
  • Improved handling of queries involving unavailable or unmatched tree fields.
  • Added warnings instead of failing when tree-based fields cannot be matched.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 0139a431-32be-49b1-a1ad-38e04dbb1f1a

📥 Commits

Reviewing files that changed from the base of the PR and between eca16dc and 8c0fa8d.

📒 Files selected for processing (3)

• specifyweb/backend/stored_queries/execution.py
• specifyweb/backend/stored_queries/query_construct.py
• specifyweb/backend/stored_queries/queryfieldspec.py

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

Stored query handling now tolerates unmapped fields and missing tree ranks. Tree joins can be reverted, missing ORM fields skip filter processing, and displayed fields receive NULL result columns to preserve output alignment.

Changes

Stored query resilience

Layer / File(s)	Summary
Tree rank fallback specifyweb/backend/stored_queries/query_construct.py	handle_tree_field saves the pre-join query and returns it when no tree rank exists. The method logs the missing rank instead of raising an assertion.
Missing field propagation specifyweb/backend/stored_queries/queryfieldspec.py, specifyweb/backend/stored_queries/execution.py	add_to_query stops when no ORM field exists. Displayed fields add a SQL NULL column before filter and sort processing is skipped.

Suggested reviewers: carolinedenis

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 8c0fa

Stored queries with unmapped fields or missing tree ranks now avoid execution errors while preserving output alignment and skipping unavailable filters and sorts. No concrete merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 4 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Automatic Tests	⚠️ Warning	Automatic tests are necessary for this behavior change. The authoritative PR diff changes only execution.py, query_construct.py, and queryfieldspec.py; it changes 3 files with no test-path chang…	Add automatic regression tests in specifyweb/backend/stored_queries/tests. Cover a missing displayed tree-rank field and assert that the result contains a NULL placeholder in the original column position. Cover a missing tree-rank field…
Testing Instructions	⚠️ Warning	The instructions cover table-view execution, filter/sort handling, and export behavior, but they omit the Statistics component. The changed build_query path is also used by `/stored_query/ephemeral/…	Update the instructions to include importing or opening the affected query in Statistics and confirm that the statistic loads without an error. State how to obtain or import the invalid query, because the normal query builder may not allow …

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The changes address issue #3351 by handling unavailable tree-rank fields without raising errors. Displayed fields retain column positions, and missing filter or sort fields are skipped.
Out of Scope Changes check	✅ Passed	All changes are within scope. They modify query execution, tree-rank handling, and field processing to support graceful handling of missing tree ranks.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main change: queries now skip unavailable tree ranks.

Full details: Automatic Tests

Explanation

Automatic tests are necessary for this behavior change. The authoritative PR diff changes only execution.py, query_construct.py, and queryfieldspec.py; it changes 3 files with no test-path changes. The repository contains stored-query execution and CSV export test suites, but the PR adds no test for a missing displayed tree rank or for a missing filtered/sorted tree rank. This also conflicts with the PR description, which claims that such automated coverage exists.

Resolution

Add automatic regression tests in specifyweb/backend/stored_queries/tests. Cover a missing displayed tree-rank field and assert that the result contains a NULL placeholder in the original column position. Cover a missing tree-rank field used for filtering or sorting and assert that execution succeeds, ignores that field, and preserves the result set. Include export coverage if the changed export path is part of the intended fix.

Full details: Testing Instructions

Explanation

The instructions cover table-view execution, filter/sort handling, and export behavior, but they omit the Statistics component. The changed build_query path is also used by /stored_query/ephemeral/, which the Statistics frontend calls. This omission matters because issue #3351 reports failures when imported queries run in Statistics. The description also claims automated coverage, but the authoritative PR diff changes only three backend files and adds no tests or missing-rank test references.

Resolution

Update the instructions to include importing or opening the affected query in Statistics and confirm that the statistic loads without an error. State how to obtain or import the invalid query, because the normal query builder may not allow creation of an unavailable rank. Specify CSV export, and test other supported export modes if they are in scope. Verify that a missing displayed rank produces an empty placeholder and preserves later columns, while a missing filtered or sorted rank does not change the result set. Add the claimed automated tests or remove the claim about automated coverage.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch issue-3351

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[g1rly-c0d3r]

Testing instructions

• Open or create a stored query containing a tree-rank field that is not present in the active collection. (If you are having a hard time finding one, I guess you can use this: FakeRankQuery.json)
• Execute the query in the table view and confirm that it does not crash and that the remaining result columns stay aligned with their headers.
• Apply a filter or sort using the unavailable rank and confirm that the query still executes while ignoring that field.
• Export the same query and confirm that the export completes and the unavailable field is omitted or empty without shifting other columns.

---

Everything looked good, none of the columns were shifted and no errors happened. I looked at the re-exported query and the fake rank was still in the export, but the query results were empty. I'm not sure if you meant that in the exported JSON the fake rank should be omitted.

[JDAM2k4]

Testing instructions

• Open or create a stored query containing a tree-rank field that is not present in the active collection. (If you are having a hard time finding one, I guess you can use this: FakeRankQuery.json)
• Execute the query in the table view and confirm that it does not crash and that the remaining result columns stay aligned with their headers.
• Apply a filter or sort using the unavailable rank and confirm that the query still executes while ignoring that field.
• Export the same query and confirm that the export completes and the unavailable field is omitted or empty without shifting other columns.

---

Seems good to me. This may also fix #3369 but I'm not sure.