Support free-threaded CPython in native extensions

[Borda]

Summary

Enable faster_coco_eval to load on free-threaded CPython without re-enabling the GIL for the entire process.

• Declare both pybind11 extension modules safe for free-threaded interpreters.
• Synchronize the mutable native Dataset cache and avoid exposing cache references after their lock is released.
• Require the first pybind11 release that provides mod_gil_not_used.
• Run the normal installed-sdist suite on CPython 3.10-3.14 across Linux, macOS, and Windows, plus no-GIL gates on 3.13t and 3.14t.

Why this matters

Before this change, importing either native extension on CPython 3.14t warns and enables the GIL process-wide. Evaluation results remain correct, but other Python threads lose free-threaded execution. With this change, the GIL remains disabled after both extensions are imported.

The expected benefit is concurrent throughput, not lower latency for one COCO evaluation.

Python 3.14t benchmark

CPython 3.14.2 free-threaded on macOS arm64. Each row uses two warmups and 15 alternating fresh-process base/PR pairs. The bbox workload contains 200 images and 10,000 detections with maxDets=[1, 10, 50]; each worker owns an independent evaluator.

Concurrent evaluators	Base	PR	Throughput gain	Paired 95% interval	Favorable pairs
1	14.06 eval/s	14.20 eval/s	1.01x	0.995-1.028x	9/15
2	11.68 eval/s	19.01 eval/s	1.61x	1.49-1.66x	15/15
4	7.69 eval/s	29.07 eval/s	3.76x	3.54-4.00x	15/15

Four concurrent evaluations complete in 0.13760 seconds with the PR versus 0.52048 seconds on the base revision. Every run produced the same exact result digest.

Standard CPython 3.13.15 also produced the same digest. An extended alternating A/B run measured a 0.91% total-time change and a -0.03% peak-RSS change, so the synchronization cost remained below the predeclared 3% regression guard.

Benchmark revisions:

• Base: 123cc73a1da42420d77e230c9d02cba8b4d83ac3
• Measured implementation: b67e5d163c0ee350051962e8f1351188a0ddeec0

Safety and reliability

• All mutable Dataset containers and cache versions are mutex-protected.
• Python conversions and Python reference destruction occur outside the mutex, preventing callback/finalizer re-entry deadlocks.
• Cache misses publish parsed data only if the annotation version still matches the source snapshot.
• Cached annotations return by value so concurrent eviction cannot leave a dangling reference.
• CI runs the standard sdist suite on CPython 3.10-3.14 across three operating systems. Separate 3.13t/3.14t jobs import outside the checkout, treat warnings as errors, check that the GIL remains disabled, and run the contention regressions.

Verification

• CPython 3.14.2t: both extensions imported with the GIL still disabled.
• CPython 3.14.2t: 16 passed in test_cpp_safety_regressions.py.
• CPython 3.10.11: full suite 211 passed, 2 skipped.
• C++ formatting hook: passed on the focused evaluator-module change.
• Hosted standard 3.14 and free-threaded 3.13t/3.14t jobs: pending until this PR revision runs in GitHub Actions.

Claim boundary

This PR does not make one evaluation materially faster and does not claim universal linear scaling, shared-mutable-Dataset scaling, or cross-platform equivalence. The measured 1.61-3.76x gain applies to 2-4 independent evaluators on one synthetic bbox workload and one macOS arm64 host.

[Copilot]

🟡 Changes recommended

Mutex-protected Python callbacks can deadlock, cache invalidation is incomplete, and the claimed pickle path lacks coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Declares both native modules compatible with free-threaded CPython and synchronizes shared dataset state.

Changes:

• Adds mod_gil_not_used declarations and Python 3.14 metadata.
• Adds dataset locking, safe cache returns, and explicit move operations.
• Adds concurrency regression tests and raises the pybind11 minimum version.

File summaries

File	Description
csrc/faster_eval_api/coco_eval/dataset.cpp	Implements synchronization and move behavior.
csrc/faster_eval_api/coco_eval/dataset.h	Defines mutex-backed dataset API.
csrc/faster_eval_api/faster_eval_api.cpp	Declares free-threading support.
csrc/mask_api/mask_api.cpp	Declares free-threading support.
pyproject.toml	Raises pybind11 build requirement.
setup.py	Adds Python 3.14 classifier.
tests/test_cpp_safety_regressions.py	Adds concurrent cache tests.

Review details

• Files reviewed: 7/7 changed files
• Comments generated: 6
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Move assignment is missing, the promised single-lock sweep is absent, and merged annotations incur an avoidable extra copy.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

csrc/faster_eval_api/coco_eval/dataset.cpp:299

• This makes the returned vector const and then copies every InstanceAnnotation into merged. Since get_cpp_annotations now returns an owned vector, this branch can still move its elements as before; otherwise the API change adds a second O(number of annotations) copy for every merged category sweep.

                                const std::vector<InstanceAnnotation> anns =
                                    get_cpp_annotations(img_id, cat_id);
                                merged.insert(merged.end(), anns.begin(),
                                              anns.end());

• Files reviewed: 8/8 changed files
• Comments generated: 2
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The pybind11 cap excludes Python 3.14 support, and free-threaded CI does not exercise concurrent evaluator or mask operations.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 7/7 changed files
• Comments generated: 2
• Review effort level: Balanced

[espressolee]

I independently validated exact base 123cc73a1da42420d77e230c9d02cba8b4d83ac3 against exact head ae7242631b374075c6c7f2a134a8c79f9e18095a on macOS arm64 with CPython 3.14.6 and 3.14.6t.

The primary compatibility result checks out: the base extension re-enables the GIL on 3.14t, while this head imports both native modules with the GIL still disabled. The head passed tests/test_cpp_safety_regressions.py (17/17) and the full local suite (177 passed, 28 skipped) on both regular and free-threaded 3.14.6. As positive controls, the two new stale-cache/append-invalidation tests fail against the base and pass against the head; the exact-head hosted matrix is also green.

I found one blocking liveness regression in the cache-publication retry. annotation_version is global to the dataset, so an append to an unrelated key invalidates the snapshot for the key currently being converted. Because conversion intentionally runs arbitrary Python hooks outside the mutex, a conversion hook that appends to a different key can invalidate every attempt and make get_cpp_annotations() retry forever.

A subprocess reproducer against the exact head timed out in 5/5 runs on both 3.14.6 and 3.14.6t. The same scenario returned in 5/5 runs on the base, and a one-shot mutation control returned in 5/5 runs on the head, so this is repeated global invalidation rather than ordinary conversion latency. Minimal shape:

dataset = _eval.Dataset()

class Area:
    def __float__(self):
        dataset.append_ref(
            999, 1, {"id": 2, "area": 1.0, "iscrowd": 0}
        )
        return 10.0

dataset.append_ref(
    1, 1, {"id": 1, "area": Area(), "iscrowd": 0}
)
dataset.get_cpp_annotations(1, 1)  # does not return at this head

The retry is at csrc/faster_eval_api/coco_eval/dataset.cpp:250: it compares the snapshot against a dataset-wide generation that every append_ref() increments.

A scratch proof using a global replacement epoch for clean/load_tuple plus per-key append versions makes both the repeated-hook reproducer and the one-shot control return, while retaining 17/17 focused tests and 177 passed / 28 skipped in both interpreter modes. An equivalent target-key identity/version check would also address the issue.

Continuous mutation of the requested key may reasonably require retry/abstention semantics; mutation of an unrelated key should not be able to starve this read. My local scope was macOS arm64; the PR's exact-head hosted cross-platform checks are green.

[MiXaiLL76]

Suggested change

	run: python -m pytest -q test_cpp_safety_regressions.py -k "TestDatasetConcurrency or test_concurrent_evaluator_and_mask_operations"
	run: python -m pytest -q test_cpp_safety_regressions.py -k "TestDatasetConcurrency"

[MiXaiLL76]

Suggested change

	"pybind11>=3.0.0, <4",
	"pybind11>=2.13.0; python_version<'3.14'",
	"pybind11>=3.0.0; python_version>='3.14'"

[MiXaiLL76]

Suggested change

	const auto& key = kv.first;
	const auto& ann_list = kv.second;

[MiXaiLL76]

@Borda
dataset.cpp
dataset.h via svg

#128