Skip to content

perf(coco): clone indexed COCO instead of rehashing in COCOeval ctor - #18

Open
Borda wants to merge 1 commit into
derekallman:mainfrom
Borda:perf/D
Open

Borda wants to merge 1 commit into
derekallman:mainfrom
Borda:perf/D

Conversation

@Borda

@Borda Borda commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

This pull request optimizes the construction of the COCOeval evaluator by having it clone an already-indexed COCO object instead of rebuilding the index from the dataset each time. This change significantly improves performance for large datasets, ensures warnings are not redundantly reprinted, and maintains correctness by verifying that the clone is a faithful, independent copy. The most important changes are as follows:

Performance and Correctness Improvements

  • The COCOeval constructor now clones the fully-indexed COCO objects for both ground-truth and detection inputs, rather than rebuilding the index from the raw dataset. This results in a 74% faster constructor and an 11% faster end-to-end evaluation on large workloads, with no increase in memory usage. Warnings from index creation are no longer redundantly reprinted. [1] [2]

Codebase Enhancements

  • The COCO struct in hotcoco now derives Clone, enabling efficient deep copies of the entire object, including its index maps and warnings.

Testing and Validation

  • Added a comprehensive test, test_clone_is_a_faithful_reindex, to ensure that cloning a COCO object produces an exact and independent copy of all index maps and warnings, and that further mutations to the clone do not affect the original.

Documentation

  • Updated the changelog to describe the new cloning behavior in COCOeval and its impact on performance, memory usage, and warning handling.
  • Minor formatting fix in the changelog for improved readability.

  • COCO derives Clone (additive, private fields unchanged)
  • PyCOCOeval::new copies coco_gt/coco_dt.inner directly instead of cloning the dataset and rebuilding the index via from_dataset
  • safe because every PyCOCO write path (dataset setter, derived*, without_image_dir) rebuilds the whole COCO, so inner's index is never stale
  • side effect: duplicate-id/unnamed-category warnings no longer reprint to stderr on every COCOeval() construction, since the clone carries the already-collected warnings instead of regenerating them
  • measured on a 1.5M-annotation RF-DETR-shaped workload: ctor -74%, end-to-end -11%; RSS unchanged (still a full copy)
  • 10/10 bit-identical vs main across ten configs
  • new test_clone_is_a_faithful_reindex fails on 2 injected defects (a hand-written Clone dropping img_cat_to_anns or warnings)

- COCO derives Clone (additive, private fields unchanged)
- PyCOCOeval::new copies coco_gt/coco_dt.inner directly instead of
  cloning the dataset and rebuilding the index via from_dataset
- safe because every PyCOCO write path (dataset setter, derived*,
  without_image_dir) rebuilds the whole COCO, so inner's index is
  never stale
- side effect: duplicate-id/unnamed-category warnings no longer
  reprint to stderr on every COCOeval() construction, since the
  clone carries the already-collected warnings instead of
  regenerating them
- measured on a 1.5M-annotation RF-DETR-shaped workload: ctor -74%,
  end-to-end -11%; RSS unchanged (still a full copy)
- 10/10 bit-identical vs main across ten configs
- new test_clone_is_a_faithful_reindex fails on 2 injected defects
  (a hand-written Clone dropping img_cat_to_anns or warnings)

---
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@Borda Borda changed the title perf(coco): clone the indexed COCO instead of rehashing in COCOeval ctor perf(coco): clone indexed COCO instead of rehashing in COCOeval ctor Sep 25, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant