Adding support for subinterpreters

[mwaddoups]

Currently, pyzmq does not support subinterpreters (from concurrent.subinterpreters) - these were introduced in Python 3.12 and made easier to access in Python 3.14. These allow for full context isolation, and since the interpreters don't share a GIL they can use separate cores when threaded without being in free-threaded mode.

The import just fails as the C extensions have not been marked as supporting subinterpreters. An example simple test script below.

from concurrent import interpreters
from contextlib import closing


def echo_responder(ctx_addr: int):
    import zmq

    shadowd_ctx = zmq.Context.shadow(ctx_addr)
    with shadowd_ctx.socket(zmq.DEALER) as rep_sock:
        rep_sock.bind("inproc://test")
        rep_sock.setsockopt(zmq.RCVTIMEO, 500)
        while input := rep_sock.recv():
            rep_sock.send(input)


import zmq

with closing(interpreters.create()) as responder:
    with zmq.Context() as ctx:
        sock = ctx.socket(zmq.DEALER)
        sock.connect("inproc://test")

        t = responder.call_in_thread(echo_responder, ctx.underlying)

        for _ in range(100):
            sock.send(b"DATA")
            sock.recv()

        sock.send(b"")
        t.join()

This raises

ImportError: module zmq.backend.cython._zmq does not support loading in subinterpreters

However, I have been able to recompile ZMQ and get thing to work.

The requirements for module isolation are noted here , but Cython has experimental support for enabling module state by CFLAGS.

I made the following changes

• Add # subinterpreters_compatible: own_gil to the zmq Cython backend header.
• Add CYTHON_USE_MODULE_STATE=1 as a CFLAG. This is noted as experimental, but mainly that it doesn't move global cdefs into state just yet.
• Add CYTHON_USE_TYPE_SPECS=1 as a CFLAG. This I think is needed because of the use of cdef class, and without this the module imports but errors on import.

However, with these 3 changes the above code works and is able to share a context.

I'm happy to submit a PR for the above - but I'm not sure what's preferred from the project. Since some of this seems to be experimental on cython side, would it be better to have as an opt-in feature (e.g. PYZMQ_CYTHON_SUBINTERPRETERS)?

I'm also not an expert on these matters, so there may be more to this!

Thanks for your work on a great project.

[mwaddoups]

Whoops, somehow I missed this PR from @minrk until just now: #2136 - I guess its the CYTHON_USE_TYPE_SPECS=1 flag that will make things work?

[minrk]

@mwaddoups please feel free to open your own PR! I added a test in #2136 if you want to copy that.