Skip to content

perf(coco): fuse load_res_anns into one validate/derive/id pass - #19

Open
Borda wants to merge 1 commit into
derekallman:mainfrom
Borda:perf/E
Open

Borda wants to merge 1 commit into
derekallman:mainfrom
Borda:perf/E

Conversation

@Borda

@Borda Borda commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

This pull request significantly improves the performance and clarity of the load_res_anns method in the COCO Rust implementation by validating, deriving geometry, and assigning IDs in a single pass over the detections. It also introduces new regression tests to ensure correct warning behavior and removes unnecessary code. These changes lead to a measurable speedup for large workloads and maintain bit-identical evaluation results.

Performance and Validation Improvements

  • The load_res_anns function now validates, derives geometry, and assigns IDs in a single loop over the detection annotations, instead of five separate passes. It checks ground-truth (GT) membership using the existing index rather than rebuilding a HashSet each time, resulting in a 16–19% speedup on large workloads. The method also ensures only one warning per mismatch kind (image/category), naming the first offender, and still immediately rejects NaN scores. [1] [2] [3]

Codebase Simplification

  • The now-redundant validate_results method has been removed, with its logic fused into the main loop of load_res_anns. This reduces code duplication and improves maintainability.

Testing Enhancements

  • Two new regression tests have been added:
    • load_res_anns_warns_once_per_mismatch_kind ensures that only one warning is issued per mismatch type and that the first offending annotation is reported.
    • load_res_anns_skips_category_check_when_gt_has_no_categories ensures that no category mismatch warnings are emitted if the GT set has no categories.

Minor Cleanups

  • Unused imports (HashSet) have been removed from coco.rs.
  • Minor formatting and documentation improvements in the changelog for clarity.

These changes collectively make the code faster, clearer, and better tested, especially for large-scale detection evaluation workloads.


  • validate_results, geometry derive, and id assignment merged into a single loop over detections instead of five separate passes
  • image_id/category_id GT membership now probes self.imgs/self.cats (the index this COCO already built) instead of rebuilding a fresh HashSet of GT ids on every load_res_anns call
  • NaN score rejection, derive_from_bbox's rectangular polygon, the unconditional id reassignment, and once-per-kind mismatch warnings are all preserved in content, only fused into fewer passes
  • both the in-memory dict path and the numpy loadRes(ndarray) fast path share load_res_anns, so both benefit from one fix
  • measured on a 1.5M-annotation RF-DETR-shaped workload: loadRes -16% to -19% (non-overlapping ranges across two rep-count runs); end-to-end effect not distinguishable from run-to-run noise
  • 10/10 bit-identical vs main across ten configs
  • new tests load_res_anns_warns_once_per_mismatch_kind and load_res_anns_skips_category_check_when_gt_has_no_categories fail on 3 injected defects (dropped warn-once guard on each mismatch kind, dropped has_cats gate)

- validate_results, geometry derive, and id assignment merged into a
  single loop over detections instead of five separate passes
- image_id/category_id GT membership now probes self.imgs/self.cats
  (the index this COCO already built) instead of rebuilding a fresh
  HashSet of GT ids on every load_res_anns call
- NaN score rejection, derive_from_bbox's rectangular polygon, the
  unconditional id reassignment, and once-per-kind mismatch warnings
  are all preserved in content, only fused into fewer passes
- both the in-memory dict path and the numpy loadRes(ndarray) fast
  path share load_res_anns, so both benefit from one fix
- measured on a 1.5M-annotation RF-DETR-shaped workload: loadRes
  -16% to -19% (non-overlapping ranges across two rep-count runs);
  end-to-end effect not distinguishable from run-to-run noise
- 10/10 bit-identical vs main across ten configs
- new tests load_res_anns_warns_once_per_mismatch_kind and
  load_res_anns_skips_category_check_when_gt_has_no_categories fail
  on 3 injected defects (dropped warn-once guard on each mismatch
  kind, dropped has_cats gate)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant