Merge branch 'main' into docs/pep750-first-pass

Author: davepeck

.github/CODEOWNERS

@@ -281,9 +281,13 @@ Doc/howto/clinic.rst          @erlend-aasland
 # Subinterpreters
 **/*interpreteridobject.*     @ericsnowcurrently
 **/*crossinterp*              @ericsnowcurrently
-Lib/test/support/interpreters/  @ericsnowcurrently
 Modules/_interp*module.c      @ericsnowcurrently
+Lib/test/test__interp*.py     @ericsnowcurrently
+Lib/concurrent/interpreters/  @ericsnowcurrently
+Lib/test/support/channels.py  @ericsnowcurrently
+Doc/library/concurrent.interpreters.rst  @ericsnowcurrently
 Lib/test/test_interpreters/   @ericsnowcurrently
+Lib/concurrent/futures/interpreter.py  @ericsnowcurrently
 
 # Android
 **/*Android*                  @mhsmith @freakboy3742

.gitignore

@@ -131,6 +131,7 @@ Tools/unicode/data/
 /autom4te.cache
 /build/
 /builddir/
+/compile_commands.json
 /config.cache
 /config.log
 /config.status

Doc/c-api/init.rst

@@ -492,17 +492,8 @@ Initializing and finalizing the interpreter
    strings other than those passed in (however, the contents of the strings
    pointed to by the argument list are not modified).
 
-   The return value will be ``0`` if the interpreter exits normally (i.e.,
-   without an exception), ``1`` if the interpreter exits due to an exception,
-   or ``2`` if the argument list does not represent a valid Python command
-   line.
-
-   Note that if an otherwise unhandled :exc:`SystemExit` is raised, this
-   function will not return ``1``, but exit the process, as long as
-   ``Py_InspectFlag`` is not set. If ``Py_InspectFlag`` is set, execution will
-   drop into the interactive Python prompt, at which point a second otherwise
-   unhandled :exc:`SystemExit` will still exit the process, while any other
-   means of exiting will set the return value as described above.
+   The return value is ``2`` if the argument list does not represent a valid
+   Python command line, and otherwise the same as :c:func:`Py_RunMain`.
 
    In terms of the CPython runtime configuration APIs documented in the
    :ref:`runtime configuration <init-config>` section (and without accounting
@@ -539,23 +530,18 @@ Initializing and finalizing the interpreter
 
    If :c:member:`PyConfig.inspect` is not set (the default), the return value
    will be ``0`` if the interpreter exits normally (that is, without raising
-   an exception), or ``1`` if the interpreter exits due to an exception. If an
-   otherwise unhandled :exc:`SystemExit` is raised, the function will immediately
-   exit the process instead of returning ``1``.
+   an exception), the exit status of an unhandled :exc:`SystemExit`, or ``1``
+   for any other unhandled exception.
 
    If :c:member:`PyConfig.inspect` is set (such as when the :option:`-i` option
    is used), rather than returning when the interpreter exits, execution will
    instead resume in an interactive Python prompt (REPL) using the ``__main__``
    module's global namespace. If the interpreter exited with an exception, it
    is immediately raised in the REPL session. The function return value is
-   then determined by the way the *REPL session* terminates: returning ``0``
-   if the session terminates without raising an unhandled exception, exiting
-   immediately for an unhandled :exc:`SystemExit`, and returning ``1`` for
-   any other unhandled exception.
-
-   This function always finalizes the Python interpreter regardless of whether
-   it returns a value or immediately exits the process due to an unhandled
-   :exc:`SystemExit` exception.
+   then determined by the way the *REPL session* terminates: ``0``, ``1``, or
+   the status of a :exc:`SystemExit`, as specified above.
+
+   This function always finalizes the Python interpreter before it returns.
 
    See :ref:`Python Configuration <init-python-config>` for an example of a
    customized Python that always runs in isolated mode using

Doc/c-api/typeobj.rst

@@ -686,6 +686,26 @@ and :c:data:`PyType_Type` effectively act as defaults.)
    instance, and call the type's :c:member:`~PyTypeObject.tp_free` function to
    free the object itself.
 
+   If you may call functions that may set the error indicator, you must use
+   :c:func:`PyErr_GetRaisedException` and :c:func:`PyErr_SetRaisedException`
+   to ensure you don't clobber a preexisting error indicator (the deallocation
+   could have occurred while processing a different error):
+
+   .. code-block:: c
+
+     static void
+     foo_dealloc(foo_object *self)
+     {
+         PyObject *et, *ev, *etb;
+         PyObject *exc = PyErr_GetRaisedException();
+         ...
+         PyErr_SetRaisedException(exc);
+     }
+
+   The dealloc handler itself must not raise an exception; if it hits an error
+   case it should call :c:func:`PyErr_FormatUnraisable` to log (and clear) an
+   unraisable exception.
+
    No guarantees are made about when an object is destroyed, except:
 
    * Python will destroy an object immediately or some time after the final

Doc/c-api/unicode.rst

@@ -1827,7 +1827,7 @@ object.
    On success, return ``0``.
    On error, set an exception, leave the writer unchanged, and return ``-1``.
 
-   .. versionadded:: next
+   .. versionadded:: 3.14
 
 .. c:function:: int PyUnicodeWriter_WriteWideChar(PyUnicodeWriter *writer, const wchar_t *str, Py_ssize_t size)
 

Doc/library/calendar.rst

@@ -251,7 +251,7 @@ interpreted as prescribed by the ISO 8601 standard.  Year 0 is 1 BC, year -1 is
       3) specifies the number of months per row. *css* is the name for the
       cascading style sheet to be used. :const:`None` can be passed if no style
       sheet should be used. *encoding* specifies the encoding to be used for the
-      output (defaulting to the system default encoding).
+      output (defaulting to ``'utf-8'``).
 
 
    .. method:: formatmonthname(theyear, themonth, withyear=True)

Doc/library/concurrency.rst

@@ -18,6 +18,7 @@ multitasking). Here's an overview:
    multiprocessing.shared_memory.rst
    concurrent.rst
    concurrent.futures.rst
+   concurrent.interpreters.rst
    subprocess.rst
    sched.rst
    queue.rst

Doc/library/concurrent.interpreters.rst

@@ -0,0 +1,198 @@
+:mod:`!concurrent.interpreters` --- Multiple interpreters in the same process
+=============================================================================
+
+.. module:: concurrent.interpreters
+   :synopsis: Multiple interpreters in the same process
+
+.. moduleauthor:: Eric Snow <ericsnowcurrently@gmail.com>
+.. sectionauthor:: Eric Snow <ericsnowcurrently@gmail.com>
+
+.. versionadded:: 3.14
+
+**Source code:** :source:`Lib/concurrent/interpreters.py`
+
+--------------
+
+
+Introduction
+------------
+
+The :mod:`!concurrent.interpreters` module constructs higher-level
+interfaces on top of the lower level :mod:`!_interpreters` module.
+
+.. XXX Add references to the upcoming HOWTO docs in the seealso block.
+
+.. seealso::
+
+   :ref:`isolating-extensions-howto`
+       how to update an extension module to support multiple interpreters
+
+   :pep:`554`
+
+   :pep:`734`
+
+   :pep:`684`
+
+.. XXX Why do we disallow multiple interpreters on WASM?
+
+.. include:: ../includes/wasm-notavail.rst
+
+
+Key details
+-----------
+
+Before we dive into examples, there are a small number of details
+to keep in mind about using multiple interpreters:
+
+* isolated, by default
+* no implicit threads
+* not all PyPI packages support use in multiple interpreters yet
+
+.. XXX Are there other relevant details to list?
+
+In the context of multiple interpreters, "isolated" means that
+different interpreters do not share any state.  In practice, there is some
+process-global data they all share, but that is managed by the runtime.
+
+
+Reference
+---------
+
+This module defines the following functions:
+
+.. function:: list_all()
+
+   Return a :class:`list` of :class:`Interpreter` objects,
+   one for each existing interpreter.
+
+.. function:: get_current()
+
+   Return an :class:`Interpreter` object for the currently running
+   interpreter.
+
+.. function:: get_main()
+
+   Return an :class:`Interpreter` object for the main interpreter.
+
+.. function:: create()
+
+   Initialize a new (idle) Python interpreter
+   and return a :class:`Interpreter` object for it.
+
+
+Interpreter objects
+^^^^^^^^^^^^^^^^^^^
+
+.. class:: Interpreter(id)
+
+   A single interpreter in the current process.
+
+   Generally, :class:`Interpreter` shouldn't be called directly.
+   Instead, use :func:`create` or one of the other module functions.
+
+   .. attribute:: id
+
+      (read-only)
+
+      The interpreter's ID.
+
+   .. attribute:: whence
+
+      (read-only)
+
+      A string describing where the interpreter came from.
+
+   .. method:: is_running()
+
+      Return ``True`` if the interpreter is currently executing code
+      in its :mod:`!__main__` module and ``False`` otherwise.
+
+   .. method:: close()
+
+      Finalize and destroy the interpreter.
+
+   .. method:: prepare_main(ns=None, **kwargs)
+
+      Bind "shareable" objects in the interpreter's
+      :mod:`!__main__` module.
+
+   .. method:: exec(code, /, dedent=True)
+
+      Run the given source code in the interpreter (in the current thread).
+
+   .. method:: call(callable, /, *args, **kwargs)
+
+      Return the result of calling running the given function in the
+      interpreter (in the current thread).
+
+   .. method:: call_in_thread(callable, /, *args, **kwargs)
+
+      Run the given function in the interpreter (in a new thread).
+
+Exceptions
+^^^^^^^^^^
+
+.. exception:: InterpreterError
+
+   This exception, a subclass of :exc:`Exception`, is raised when
+   an interpreter-related error happens.
+
+.. exception:: InterpreterNotFoundError
+
+   This exception, a subclass of :exc:`InterpreterError`, is raised when
+   the targeted interpreter no longer exists.
+
+.. exception:: ExecutionFailed
+
+   This exception, a subclass of :exc:`InterpreterError`, is raised when
+   the running code raised an uncaught exception.
+
+   .. attribute:: excinfo
+
+      A basic snapshot of the exception raised in the other interpreter.
+
+.. XXX Document the excinfoattrs?
+
+.. exception:: NotShareableError
+
+   This exception, a subclass of :exc:`TypeError`, is raised when
+   an object cannot be sent to another interpreter.
+
+
+.. XXX Add functions for communicating between interpreters.
+
+
+Basic usage
+-----------
+
+Creating an interpreter and running code in it::
+
+    from concurrent import interpreters
+
+    interp = interpreters.create()
+
+    # Run in the current OS thread.
+
+    interp.exec('print("spam!")')
+
+    interp.exec("""if True:
+        print('spam!')
+        """)
+
+    from textwrap import dedent
+    interp.exec(dedent("""
+        print('spam!')
+        """))
+
+    def run():
+        print('spam!')
+
+    interp.call(run)
+
+    # Run in new OS thread.
+
+    t = interp.call_in_thread(run)
+    t.join()
+
+
+.. XXX Explain about object "sharing".

Doc/library/concurrent.rst

@@ -1,6 +1,7 @@
 The :mod:`!concurrent` package
 ==============================
 
-Currently, there is only one module in this package:
+This package contains the following modules:
 
 * :mod:`concurrent.futures` -- Launching parallel tasks
+* :mod:`concurrent.interpreters` -- Multiple interpreters in the same process

Doc/library/csv.rst

@@ -609,7 +609,7 @@ A slightly more advanced use of the reader --- catching and reporting errors::
            for row in reader:
                print(row)
        except csv.Error as e:
-           sys.exit('file {}, line {}: {}'.format(filename, reader.line_num, e))
+           sys.exit(f'file {filename}, line {reader.line_num}: {e}')
 
 And while the module doesn't directly support parsing strings, it can easily be
 done::