Skip to content

perf(matching): skip wasted match allocations - #16

Closed
Borda wants to merge 1 commit into
derekallman:mainfrom
Borda:perf/B
Closed

Borda wants to merge 1 commit into
derekallman:mainfrom
Borda:perf/B

Conversation

@Borda

@Borda Borda commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

This pull request optimizes the evaluation logic in the detection matching process to avoid unnecessary computation for cells that will be discarded, and adds a targeted test to ensure this logic is correct. The main improvement is that cells with no relevant ground truth or detections are now skipped before any costly allocation or matching work is performed, which should improve performance without changing evaluation results.

Evaluation logic optimization:

  • The omission gate in evaluate_cell() now runs before any match allocations or computations, skipping cells with no non-ignored detections or ground truths and no raw ground-truth IDs, thus avoiding unnecessary work. This preserves bit-identical outputs for precision, recall, scores, and stats. [1] [2] [3]

Testing:

  • Added test_evaluate_cell_gate_reads_pre_match_state to verify that the omission gate logic is correct and only reads pre-match state, ensuring that cells are kept or discarded as intended in different scenarios.

Documentation:

  • Updated the changelog to describe the new evaluation logic and its effects, including details about the new test and the unchanged outputs. [1] [2]

  • Move evaluate_cell's omission gate before match_cell(): the gate reads only partition_gt/area_filter_dt output, never match_cell's, so a discarded cell no longer pays for match_cell's five ThreshMatrix allocations and the greedy match.
  • No value changes: precision/recall/scores/stats bit-identical to main on the same 10 configs A1/A2/A3 were checked against.
  • Add test_evaluate_cell_gate_reads_pre_match_state, pinning the gate's two separate legs (has_content vs raw GT id count) with a case for each; verified it fails on an injected collapse of the gate to has_content alone.

- Move evaluate_cell's omission gate before match_cell(): the gate
  reads only partition_gt/area_filter_dt output, never match_cell's,
  so a discarded cell no longer pays for match_cell's five
  ThreshMatrix allocations and the greedy match.
- No value changes: precision/recall/scores/stats bit-identical to
  main on the same 10 configs A1/A2/A3 were checked against.
- Add test_evaluate_cell_gate_reads_pre_match_state, pinning the
  gate's two separate legs (has_content vs raw GT id count) with a
  case for each; verified it fails on an injected collapse of the
  gate to has_content alone.
---
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@derekallman

Copy link
Copy Markdown
Owner

Thanks for this one. It ends up superseded rather than merged: while chasing bit-exactness against pycocotools I found that the omission gate this PR reorders was itself the source of a divergence in eval['scores'] — pycocotools' evaluateImg skips a cell only when the raw GT and DT lists are both empty, and gather_pair already enforces exactly that, so the second gate in evaluate_cell (the has_content leg) dropped GT-less cells whose detections were all area-ignored, which pycocotools keeps as rank fillers. The fix removes that gate entirely, so there is nothing left to hoist. The other five (#17–#21) are merged; thanks for the careful measurements on each.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants