Conversation
- Move evaluate_cell's omission gate before match_cell(): the gate reads only partition_gt/area_filter_dt output, never match_cell's, so a discarded cell no longer pays for match_cell's five ThreshMatrix allocations and the greedy match. - No value changes: precision/recall/scores/stats bit-identical to main on the same 10 configs A1/A2/A3 were checked against. - Add test_evaluate_cell_gate_reads_pre_match_state, pinning the gate's two separate legs (has_content vs raw GT id count) with a case for each; verified it fails on an injected collapse of the gate to has_content alone. --- Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Owner
|
Thanks for this one. It ends up superseded rather than merged: while chasing bit-exactness against pycocotools I found that the omission gate this PR reorders was itself the source of a divergence in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request optimizes the evaluation logic in the detection matching process to avoid unnecessary computation for cells that will be discarded, and adds a targeted test to ensure this logic is correct. The main improvement is that cells with no relevant ground truth or detections are now skipped before any costly allocation or matching work is performed, which should improve performance without changing evaluation results.
Evaluation logic optimization:
evaluate_cell()now runs before any match allocations or computations, skipping cells with no non-ignored detections or ground truths and no raw ground-truth IDs, thus avoiding unnecessary work. This preserves bit-identical outputs for precision, recall, scores, and stats. [1] [2] [3]Testing:
test_evaluate_cell_gate_reads_pre_match_stateto verify that the omission gate logic is correct and only reads pre-match state, ensuring that cells are kept or discarded as intended in different scenarios.Documentation: