Merge remote-tracking branch 'origin/master' into asyncgen

Author: miracle2k

docs/source/usage.rst

@@ -34,36 +34,72 @@ After::
     
     trio_asyncio.run(async_main, *args)
 
-Equivalently, wrap your main loop in a :func:`trio_asyncio.open_loop` call ::
+Equivalently, wrap your main loop (or any other code that needs to talk to
+asyncio) in a :func:`trio_asyncio.open_loop` call ::
 
+    import trio
     import trio_asyncio
 
-    async def async_main(*args):
+    async def async_main_wrapper(*args):
         async with trio_asyncio.open_loop() as loop:
-            pass # async main code goes here
+            assert loop == asyncio.get_event_loop()
+            await async_main(*args)
+
+    trio.run(async_main_wrapper, *args)
+
+Within the ``async with`` block, an asyncio mainloop is active.
+
+As this code demonstrates, you don't need to pass the ``loop`` argument
+around, as :func:`asyncio.get_event_loop` will retrieve it when you're in
+the loop's context.
+
+.. note::
+
+   Don't do both. The following code **will not work**::
+
+        import trio
+        import trio_asyncio
 
-Within the ``async with`` block, the asyncio mainloop is active. You don't
-need to pass the ``loop`` argument around, as
-:func:`asyncio.get_event_loop` will do the right thing.
+        async def async_main_wrapper(*args):
+            async with trio_asyncio.open_loop() as loop:
+                await async_main(*args)
+
+        trio_asyncio.run(async_main_wrapper, *args)
 
 .. autofunction:: trio_asyncio.open_loop
 
 .. autofunction:: trio_asyncio.run
 
+.. note::
+
+   The ``async with open_loop()`` way of running ``trio_asyncio`` is
+   intended to transparently allow a library to use ``asyncio`` code,
+   supported by a "local" asyncio loop, without affecting the rest of your
+   Trio program.
+
+   However, currently this doesn't work because Trio does not yet support
+   ``contextvars``. Progress on this limitation is tracked in `this issue
+   on github <https://github.com/python-trio/trio-asyncio/issues/9>`_.
+
 Stopping
 --------
 
 The asyncio mainloop will be stopped automatically when the code within
-``async with open_loop()`` exits. Trio-asyncio will process all outstanding
-callbacks and terminate. As in asyncio, callbacks which are added during
-this step will be ignored.
+``async with open_loop()`` / ``trio_asyncion.run()`` exits. Trio-asyncio
+will process all outstanding callbacks and terminate. As in asyncio,
+callbacks which are added during this step will be ignored.
 
 You cannot restart the loop, nor would you want to.
 
-Asyncio main loop
-+++++++++++++++++
+Asyncio main loop.
+++++++++++++++++++
+
+Short answer: don't.
+
+.. _native-loop:
 
-Well …
+Native Mode
+-----------
 
 What you really want to do is to use a Trio main loop, and run your asyncio
 code in its context. In other words, you should transform this code::
@@ -75,35 +111,38 @@ code in its context. In other words, you should transform this code::
 to this::
 
     async def trio_main():
-        async with trio_asyncio.open_loop() as loop:
-            await loop.run_asyncio(async_main)
+        await loop.run_asyncio(async_main)
 
     def main():
-        trio.run(trio_main)
-    
-You don't need to pass around the ``loop`` argument since trio remembers it
-in its task structure: ``asyncio.get_event_loop()`` always works while
-your program is executing an ``async with open_loop():`` block.
+        trio_asyncio.run(trio_main)
 
-There is no Trio equivalent to ``loop.run_forever()``. The loop terminates
-when you leave the ``async with`` block; it cannot be halted or restarted.
+Beside this, no changes to your code are required.
 
-This mode is called an "async loop" or "asynchronous loop" because it is
-started from an async (Trio) context.
-
-Compatibility mode
+Compatibility Mode
 ------------------
 
 You still can do things "the asyncio way": the to-be-replaced code from the
-previous section still works. However, behind the scenes
-a separate thread executes the Trio main loop. It runs in lock-step with
-the thread that calls ``loop.run_forever()`` or
-``loop.run_until_complete(coro)``. Signals etc. get
-delegated if possible (except for [SIGCHLD]_). Thus, there should be no
-concurrency issues.
+:ref:`previous section <native-loop>`
+still works – or at least it attempts to work::
 
-Caveat: you may still experience problems, particularly if your code (or
-a library you're calling) does not expect to run in a different thread.
+    import asyncio
+    import trio_asyncio
+    asyncio.set_event_loop_policy(trio_asyncio.TrioPolicy())
+
+    def main():
+        loop = asyncio.get_event_loop()
+        loop.run_until_complete(async_main())
+    
+.. warning::
+
+   tl;dr: Don't use Compatibility Mode in production code.
+
+However, this is only possible because this mode starts a separate thread
+which executes the asyncio main
+loop. It runs in lock-step with the code that calls ``loop.run_forever()``
+or ``loop.run_until_complete(coro)``. Signals etc. get
+delegated if possible (except for [SIGCHLD]_). Thus, while there should be no
+concurrency issues, you may still experience hard-to-debug problems.
 
 .. [SIGCHLD] Python requires you to register SIGCHLD handlers in the main
    thread, but doesn't run them at all when waiting for another thread.
@@ -116,20 +155,23 @@ a library you're calling) does not expect to run in a different thread.
 with another call to ``loop.run_forever()`` or ``loop.run_until_complete(coro)``,
 just as with a regular asyncio loop.
 
-This mode is called a "sync loop" or "synchronous loop" because it is
-started and used from a traditional synchronous Python context.
+If you use a compatibility-mode loop in a separate thread, you *must* stop and close it
+before terminating that thread. Otherwise your thread will leak resources.
 
-If you use a sync loop in a separate thread, you *must* stop and close it
-before terminating the thread. Otherwise your thread will leak resources.
+In a multi-threaded program, globally setting the event loop policy may not
+be a good idea. If you want to run trio-asyncio in a separate thread, you
+might get away with using ``TrioPolicy().new_event_loop()`` to create a new
+event loop – but a far better idea is to use native mode.
+
+.. note::
 
-.. warning::
    Compatibility mode has been added to verify that various test suites,
-   most notably the one from asyncio itself, continue to work. In a
+   most notably the tests from asyncio itself, continue to work. In a
    real-world program with a long-running asyncio mainloop, you *really*
-   want to use a :ref:`Trio mainloop <trio-loop>` instead.
+   want to use a :ref:`native-mode main loop <native-loop>` instead.
 
-   The authors reserve the right to not fix compatibility mode bugs if that
-   would negatively impact trio-asyncio's core functions.
+   The authors reserve the right to not fix compatibility mode bugs, or
+   even to remove compatibility mode entirely.
 
 .. autoclass:: trio_asyncio.sync.SyncTrioEventLoop
 

tests/aiotest/test_timer.py

@@ -8,7 +8,7 @@ class TestTimer(aiotest.TestCase):
     @pytest.mark.trio
     async def test_display_date(self, loop):
         result = []
-        delay = 0.1
+        delay = 0.2
         count = 3
         h = trio.Event()
 

tests/conftest.py

@@ -46,3 +46,6 @@ def sync_loop():
 def pytest_pyfunc_call(pyfuncitem):
     if inspect.iscoroutinefunction(pyfuncitem.obj):
         pyfuncitem.obj = pytest.mark.trio(pyfuncitem.obj)
+
+
+asyncio.set_event_loop_policy(trio_asyncio.TrioPolicy())

trio_asyncio/adapter.py

@@ -7,36 +7,52 @@
 # import logging
 # logger = logging.getLogger(__name__)
 
-from functools import wraps
+from functools import wraps, partial
 
 __all__ = ['trio2aio', 'aio2trio']
 
 
 def trio2aio(proc):
     if isasyncgenfunction(proc):
+
         @wraps(proc)
-        def call(*args):
-            return trio_asyncio.wrap_generator(proc, *args)
+        def call(*args, **kwargs):
+            proc_ = proc
+            if kwargs:
+                proc_ = partial(proc_, **kwargs)
+            return trio_asyncio.wrap_generator(proc_, *args)
 
     else:
+
         @wraps(proc)
-        async def call(*args):
-            return await trio_asyncio.run_asyncio(proc, *args)
+        async def call(*args, **kwargs):
+            proc_ = proc
+            if kwargs:
+                proc_ = partial(proc_, **kwargs)
+            return await trio_asyncio.run_asyncio(proc_, *args)
 
     return call
 
 
 def aio2trio(proc):
+    """Decorate a Trio function so that it's callable by asyncio (only)."""
+
     @wraps(proc)
-    async def call(*args):
-        return await trio_asyncio.run_trio(proc, *args)
+    async def call(*args, **kwargs):
+        proc_ = proc
+        if kwargs:
+            proc_ = partial(proc_, **kwargs)
+        return await trio_asyncio.run_trio(proc_, *args)
 
     return call
 
 
 def aio2trio_task(proc):
     @wraps(proc)
-    async def call(*args):
-        trio_asyncio.run_trio_task(proc, *args)
+    async def call(*args, **kwargs):
+        proc_ = proc
+        if kwargs:
+            proc_ = partial(proc_, **kwargs)
+        trio_asyncio.run_trio_task(proc_, *args)
 
     return call

trio_asyncio/async_.py

@@ -106,11 +106,9 @@ def _main_loop_exit(self):
         self._thread = None
 
     async with trio.open_nursery() as nursery:
-        old_loop = asyncio.get_event_loop()
         loop = TrioEventLoop(queue_len=queue_len)
         try:
             loop._closed = False
-            asyncio.set_event_loop(loop)
             await loop._main_loop_init(nursery)
             await nursery.start(loop._main_loop)
             await yield_(loop)
@@ -120,5 +118,4 @@ def _main_loop_exit(self):
             finally:
                 await loop._main_loop_exit()
                 loop.close()
-                asyncio.set_event_loop(old_loop)
                 nursery.cancel_scope.cancel()

trio_asyncio/base.py

@@ -646,6 +646,7 @@ async def _main_loop_init(self, nursery):
         self._nursery = nursery
         self._task = trio.hazmat.current_task()
         self._token = trio.hazmat.current_trio_token()
+        asyncio.events._set_running_loop(self)
 
     async def _main_loop(self, task_status=trio.TASK_STATUS_IGNORED):
         """Run the loop by processing its event queue.
@@ -725,6 +726,7 @@ async def _main_loop_exit(self):
         # clean core fields
         self._nursery = None
         self._task = None
+        asyncio.events._set_running_loop(None)
 
     def is_running(self):
         if self._stopped is None:

trio_asyncio/loop.py

@@ -98,7 +98,7 @@ def set_event_loop(self, loop):
         # in a sub-task, which is exactly what we intend to be possible
         if self._trio_local._loop is not None and loop is not None and \
                 self._trio_local._task == task:
-            raise RuntimeError('You cannot replace an event loop.')
+            raise RuntimeError('You cannot replace an event loop.', self._trio_local._loop, loop)
         self._trio_local._loop = loop
         self._trio_local._task = task
 
@@ -178,17 +178,35 @@ def wrap_generator(proc, *args):
 
 
 async def run_asyncio(proc, *args):
+    """Run an asyncio function or method from Trio.
+
+    :return: whatever the procedure returns.
+    :raises: whatever the procedure raises.
+
+    This is a Trio coroutine.
+    """
+
     loop = asyncio.get_event_loop()
     if not isinstance(loop, TrioEventLoop):
         raise RuntimeError("Need to run in a trio_asyncio.open_loop() context")
     return await loop.run_asyncio(proc, *args)
 
 
-async def run_coroutine(fut, scope=None):
+async def run_coroutine(fut):
+    """Wait for an asyncio future/coroutine.
+
+    Cancelling the current Trio scope will cancel the future/coroutine.
+
+    Cancelling the future/coroutine will cause an
+    ``asyncio.CancelledError``.
+
+    This is a Trio coroutine.
+    """
+
     loop = asyncio.get_event_loop()
     if not isinstance(loop, TrioEventLoop):
         raise RuntimeError("Need to run in a trio_asyncio.open_loop() context")
-    return await loop.run_coroutine(fut, scope=scope)
+    return await loop.run_coroutine(fut)
 
 
 def run_trio(proc, *args):
@@ -230,6 +248,3 @@ async def _run_task(proc, args):
             return await proc(*args)
 
     trio.run(_run_task, proc, args)
-
-
-asyncio.set_event_loop_policy(TrioPolicy())