Skip to content

3.15.rst

Latest commit

 

History

History
1291 lines (920 loc) · 46 KB

3.15.rst

File metadata and controls

1291 lines (920 loc) · 46 KB

What's new in Python 3.15

Editor:Hugo van Kemenade

This article explains the new features in Python 3.15, compared to 3.14.

For full details, see the :ref:`changelog <changelog>`.

Note

Prerelease users should be aware that this document is currently in draft form. It will be updated substantially as Python 3.15 moves towards release, so it's worth checking back even after reading earlier versions.

Summary -- Release highlights

New features

PEP 799: High frequency statistical sampling profiler

A new statistical sampling profiler has been added to the new :mod:`!profiling` module as :mod:`!profiling.sampling`. This profiler enables low-overhead performance analysis of running Python processes without requiring code modification or process restart.

Unlike deterministic profilers (:mod:`cProfile` and :mod:`profile`) that instrument every function call, the sampling profiler periodically captures stack traces from running processes. This approach provides virtually zero overhead while achieving sampling rates of up to 1,000,000 Hz, making it the fastest sampling profiler available for Python (at the time of its contribution) and ideal for debugging performance issues in production environments.

Key features include:

  • Zero-overhead profiling: Attach to any running Python process without affecting its performance
  • No code modification required: Profile existing applications without restart
  • Real-time statistics: Monitor sampling quality during data collection
  • Multiple output formats: Generate both detailed statistics and flamegraph data
  • Thread-aware profiling: Option to profile all threads or just the main thread

Profile process 1234 for 10 seconds with default settings:

python -m profiling.sampling 1234

Profile with custom interval and duration, save to file:

python -m profiling.sampling -i 50 -d 30 -o profile.stats 1234

Generate collapsed stacks for flamegraph:

python -m profiling.sampling --collapsed 1234

Profile all threads and sort by total time:

python -m profiling.sampling -a --sort-tottime 1234

The profiler generates statistical estimates of where time is spent:

Real-time sampling stats: Mean: 100261.5Hz (9.97µs) Min: 86333.4Hz (11.58µs) Max: 118807.2Hz (8.42µs) Samples: 400001
Captured 498841 samples in 5.00 seconds
Sample rate: 99768.04 samples/sec
Error rate: 0.72%
Profile Stats:
      nsamples   sample%   tottime (s)    cumul%   cumtime (s)  filename:lineno(function)
      43/418858       0.0         0.000      87.9         4.189  case.py:667(TestCase.run)
    3293/418812       0.7         0.033      87.9         4.188  case.py:613(TestCase._callTestMethod)
  158562/158562      33.3         1.586      33.3         1.586  test_compile.py:725(TestSpecifics.test_compiler_recursion_limit.<locals>.check_limit)
  129553/129553      27.2         1.296      27.2         1.296  ast.py:46(parse)
      0/128129       0.0         0.000      26.9         1.281  test_ast.py:884(AST_Tests.test_ast_recursion_limit.<locals>.check_limit)
        7/67446       0.0         0.000      14.2         0.674  test_compile.py:729(TestSpecifics.test_compiler_recursion_limit)
        6/60380       0.0         0.000      12.7         0.604  test_ast.py:888(AST_Tests.test_ast_recursion_limit)
        3/50020       0.0         0.000      10.5         0.500  test_compile.py:727(TestSpecifics.test_compiler_recursion_limit)
        1/38011       0.0         0.000       8.0         0.380  test_ast.py:886(AST_Tests.test_ast_recursion_limit)
        1/25076       0.0         0.000       5.3         0.251  test_compile.py:728(TestSpecifics.test_compiler_recursion_limit)
    22361/22362       4.7         0.224       4.7         0.224  test_compile.py:1368(TestSpecifics.test_big_dict_literal)
        4/18008       0.0         0.000       3.8         0.180  test_ast.py:889(AST_Tests.test_ast_recursion_limit)
      11/17696       0.0         0.000       3.7         0.177  subprocess.py:1038(Popen.__init__)
    16968/16968       3.6         0.170       3.6         0.170  subprocess.py:1900(Popen._execute_child)
        2/16941       0.0         0.000       3.6         0.169  test_compile.py:730(TestSpecifics.test_compiler_recursion_limit)

Legend:
  nsamples: Direct/Cumulative samples (direct executing / on call stack)
  sample%: Percentage of total samples this function was directly executing
  tottime: Estimated total time spent directly in this function
  cumul%: Percentage of total samples when this function was on the call stack
  cumtime: Estimated cumulative time (including time in called functions)
  filename:lineno(function): Function location and name

Summary of Interesting Functions:

Functions with Highest Direct/Cumulative Ratio (Hot Spots):
  1.000 direct/cumulative ratio, 33.3% direct samples: test_compile.py:(TestSpecifics.test_compiler_recursion_limit.<locals>.check_limit)
  1.000 direct/cumulative ratio, 27.2% direct samples: ast.py:(parse)
  1.000 direct/cumulative ratio, 3.6% direct samples: subprocess.py:(Popen._execute_child)

Functions with Highest Call Frequency (Indirect Calls):
  418815 indirect calls, 87.9% total stack presence: case.py:(TestCase.run)
  415519 indirect calls, 87.9% total stack presence: case.py:(TestCase._callTestMethod)
  159470 indirect calls, 33.5% total stack presence: test_compile.py:(TestSpecifics.test_compiler_recursion_limit)

Functions with Highest Call Magnification (Cumulative/Direct):
  12267.9x call magnification, 159470 indirect calls from 13 direct: test_compile.py:(TestSpecifics.test_compiler_recursion_limit)
  10581.7x call magnification, 116388 indirect calls from 11 direct: test_ast.py:(AST_Tests.test_ast_recursion_limit)
  9740.9x call magnification, 418815 indirect calls from 43 direct: case.py:(TestCase.run)

The profiler automatically identifies performance bottlenecks through statistical analysis, highlighting functions with high CPU usage and call frequency patterns.

This capability is particularly valuable for debugging performance issues in production systems where traditional profiling approaches would be too intrusive.

.. seealso:: :pep:`799` for further details.

(Contributed by Pablo Galindo and László Kiss Kollár in :gh:`135953`.)

Improved error messages

  • The interpreter now provides more helpful suggestions in :exc:`AttributeError` exceptions when accessing an attribute on an object that does not exist, but a similar attribute is available through one of its members.

    For example, if the object has an attribute that itself exposes the requested name, the error message will suggest accessing it via that inner attribute:

    @dataclass
class Circle:
   radius: float

   @property
   def area(self) -> float:
      return pi * self.radius**2

class Container:
   def __init__(self, inner: Circle) -> None:
      self.inner = inner

circle = Circle(radius=4.0)
container = Container(circle)
print(container.area)

    Running this code now produces a clearer suggestion:

    Traceback (most recent call last):
File "/home/pablogsal/github/python/main/lel.py", line 42, in <module>
   print(container.area)
         ^^^^^^^^^^^^^^
AttributeError: 'Container' object has no attribute 'area'. Did you mean: 'inner.area'?

Other language changes

  • Python now uses UTF-8 as the default encoding, independent of the system's environment. This means that I/O operations without an explicit encoding, for example, open('flying-circus.txt'), will use UTF-8. UTF-8 is a widely-supported Unicode character encoding that has become a de facto standard for representing text, including nearly every webpage on the internet, many common file formats, programming languages, and more.

    This only applies when no encoding argument is given. For best compatibility between versions of Python, ensure that an explicit encoding argument is always provided. The :ref:`opt-in encoding warning <io-encoding-warning>` can be used to identify code that may be affected by this change. The special encoding='locale' argument uses the current locale encoding, and has been supported since Python 3.10.

    To retain the previous behaviour, Python's UTF-8 mode may be disabled with the :envvar:`PYTHONUTF8=0 <PYTHONUTF8>` environment variable or the :option:`-X utf8=0 <-X>` command-line option.

    .. seealso:: :pep:`686` for further details.

    (Contributed by Adam Turner in :gh:`133711`; PEP 686 written by Inada Naoki.)

  • Several error messages incorrectly using the term "argument" have been corrected. (Contributed by Stan Ulbrych in :gh:`133382`.)

  • The interpreter now tries to provide a suggestion when :func:`delattr` fails due to a missing attribute. When an attribute name that closely resembles an existing attribute is used, the interpreter will suggest the correct attribute name in the error message. For example:

    >>> class A:
...     pass
>>> a = A()
>>> a.abcde = 1
>>> del a.abcdf  # doctest: +ELLIPSIS
Traceback (most recent call last):
...
AttributeError: 'A' object has no attribute 'abcdf'. Did you mean: 'abcde'?

    (Contributed by Nikita Sobolev and Pranjal Prajapati in :gh:`136588`.)

  • Unraisable exceptions are now highlighted with color by default. This can be controlled by :ref:`environment variables <using-on-controlling-color>`. (Contributed by Peter Bierma in :gh:`134170`.)

  • The :meth:`~object.__repr__` of :class:`ImportError` and :class:`ModuleNotFoundError` now shows "name" and "path" as name=<name> and path=<path> if they were given as keyword arguments at construction time. (Contributed by Serhiy Storchaka, Oleg Iarygin, and Yoav Nir in :gh:`74185`.)

  • The :attr:`~object.__dict__` and :attr:`!__weakref__` descriptors now use a single descriptor instance per interpreter, shared across all types that need them. This speeds up class creation, and helps avoid reference cycles. (Contributed by Petr Viktorin in :gh:`135228`.)

  • The :option:`-W` option and the :envvar:`PYTHONWARNINGS` environment variable can now specify regular expressions instead of literal strings to match the warning message and the module name, if the corresponding field starts and ends with a forward slash (/). (Contributed by Serhiy Storchaka in :gh:`134716`.)

  • Functions that take timestamp or timeout arguments now accept any real numbers (such as :class:`~decimal.Decimal` and :class:`~fractions.Fraction`), not only integers or floats, although this does not improve precision. (Contributed by Serhiy Storchaka in :gh:`67795`.)

  • Added :meth:`bytearray.take_bytes(n=None, /) <bytearray.take_bytes>` to take bytes out of a :class:`bytearray` without copying. This enables optimizing code which must return :class:`bytes` after working with a mutable buffer of bytes such as data buffering, network protocol parsing, encoding, decoding, and compression. Common code patterns which can be optimized with :func:`~bytearray.take_bytes` are listed below.

    Suggested optimizing refactors

    Description

    Old

    New

    Return :class:`bytes` after working with :class:`bytearray`

    		 
    def read() -> bytes:
    buffer = bytearray(1024)
    ...
    return bytes(buffer)
    		 
    def read() -> bytes:
    buffer = bytearray(1024)
    ...
    return buffer.take_bytes()

    Empty a buffer getting the bytes

    		 
    buffer = bytearray(1024)
...
data = bytes(buffer)
buffer.clear()
    		 
    buffer = bytearray(1024)
...
data = buffer.take_bytes()

    Split a buffer at a specific separator

    		 
    buffer = bytearray(b'abc\ndef')
n = buffer.find(b'\n')
data = bytes(buffer[:n + 1])
del buffer[:n + 1]
assert data == b'abc'
assert buffer == bytearray(b'def')
    		 
    buffer = bytearray(b'abc\ndef')
n = buffer.find(b'\n')
data = buffer.take_bytes(n + 1)

    Split a buffer at a specific separator; discard after the separator

    		 
    buffer = bytearray(b'abc\ndef')
n = buffer.find(b'\n')
data = bytes(buffer[:n])
buffer.clear()
assert data == b'abc'
assert len(buffer) == 0
    		 
    buffer = bytearray(b'abc\ndef')
n = buffer.find(b'\n')
buffer.resize(n)
data = buffer.take_bytes()

    (Contributed by Cody Maloney in :gh:`139871`.)

  • Many functions related to compiling or parsing Python code, such as :func:`compile`, :func:`ast.parse`, :func:`symtable.symtable`, and :func:`importlib.abc.InspectLoader.source_to_code`, now allow the module name to be passed. It is needed to unambiguously :ref:`filter <warning-filter>` syntax warnings by module name. (Contributed by Serhiy Storchaka in :gh:`135801`.)

  • Allowed defining the __dict__ and __weakref__ :ref:`__slots__ <slots>` for any class. (Contributed by Serhiy Storchaka in :gh:`41779`.)

New modules

math.integer

This module provides access to the mathematical functions for integer arguments (PEP 791). (Contributed by Serhiy Storchaka in :gh:`81313`.)

Improved modules

argparse

calendar

collections

collections.abc

concurrent.futures

dataclasses

  • Annotations for generated __init__ methods no longer include internal type names.

dbm

difflib

functools

hashlib

  • Ensure that hash functions guaranteed to be always available exist as attributes of :mod:`hashlib` even if they will not work at runtime due to missing backend implementations. For instance, hashlib.md5 will no longer raise :exc:`AttributeError` if OpenSSL is not available and Python has been built without MD5 support. (Contributed by Bénédikt Tran in :gh:`136929`.)

http.client

http.cookies

  • Allow '"' double quotes in cookie values. (Contributed by Nick Burns and Senthil Kumaran in :gh:`92936`.)

inspect

locale

math

mimetypes

  • Add application/node MIME type for .cjs extension. (Contributed by John Franey in :gh:`140937`.)
  • Add application/toml. (Contributed by Gil Forcada in :gh:`139959`.)
  • Rename application/x-texinfo to application/texinfo. (Contributed by Charlie Lin in :gh:`140165`.)
  • Changed the MIME type for .ai files to application/pdf. (Contributed by Stan Ulbrych in :gh:`141239`.)

mmap

  • :class:`mmap.mmap` now has a trackfd parameter on Windows; if it is False, the file handle corresponding to fileno will not be duplicated. (Contributed by Serhiy Storchaka in :gh:`78502`.)

os

  • Add :func:`os.statx` on Linux kernel versions 4.11 and later with glibc versions 2.28 and later. (Contributed by Jeffrey Bosboom and Victor Stinner in :gh:`83714`.)

os.path

resource

shelve

socket

  • Add constants for the ISO-TP CAN protocol. (Contributed by Patrick Menschel and Stefan Tatschner in :gh:`86819`.)

sqlite3

ssl

sys

tarfile

timeit

tkinter

  • The :meth:`!tkinter.Text.search` method now supports two additional arguments: nolinestop which allows the search to continue across line boundaries; and strictlimits which restricts the search to within the specified range. (Contributed by Rihaan Meher in :gh:`130848`)
  • A new method :meth:`!tkinter.Text.search_all` has been introduced. This method allows for searching for all matches of a pattern using Tcl's -all and -overlap options. (Contributed by Rihaan Meher in :gh:`130848`)

types

unicodedata

unittest

venv

  • On POSIX platforms, platlib directories will be created if needed when creating virtual environments, instead of using lib64 -> lib symlink. This means purelib and platlib of virtual environments no longer share the same lib directory on platforms where :data:`sys.platlibdir` is not equal to lib. (Contributed by Rui Xi in :gh:`133951`.)

warnings

  • Improve filtering by module in :func:`warnings.warn_explicit` if no module argument is passed. It now tests the module regular expression in the warnings filter not only against the filename with .py stripped, but also against module names constructed starting from different parent directories of the filename (with /__init__.py, .py and, on Windows, .pyw stripped). (Contributed by Serhiy Storchaka in :gh:`135801`.)

xml.parsers.expat

zlib

Optimizations

csv

Removed

ctypes

glob

http.server

importlib.resources

pathlib

platform

sre_*

sysconfig

threading

  • Remove support for arbitrary positional or keyword arguments in the C implementation of :class:`~threading.RLock` objects. This was deprecated in Python 3.14. (Contributed by Bénédikt Tran in :gh:`134087`.)

typing

  • The undocumented keyword argument syntax for creating :class:`~typing.NamedTuple` classes (for example, Point = NamedTuple("Point", x=int, y=int)) is no longer supported. Use the class-based syntax or the functional syntax instead. (Contributed by Bénédikt Tran in :gh:`133817`.)

  • Using TD = TypedDict("TD") or TD = TypedDict("TD", None) to construct a :class:`~typing.TypedDict` type with zero fields is no longer supported. Use class TD(TypedDict): pass or TD = TypedDict("TD", {}) instead. (Contributed by Bénédikt Tran in :gh:`133823`.)

  • Code like class ExtraTypeVars(P1[S], Protocol[T, T2]): ... now raises a :exc:`TypeError`, because S is not listed in Protocol parameters. (Contributed by Nikita Sobolev in :gh:`137191`.)

  • Code like class B2(A[T2], Protocol[T1, T2]): ... now correctly handles type parameters order: it is (T1, T2), not (T2, T1) as it was incorrectly inferred in runtime before. (Contributed by Nikita Sobolev in :gh:`137191`.)

  • :class:`typing.ByteString` has been removed from typing.__all__. :class:`!typing.ByteString` has been deprecated since Python 3.9, and is scheduled for removal in Python 3.17.

  • The following statements now cause DeprecationWarnings to be emitted at runtime:

    • from typing import ByteString
    • import typing; typing.ByteString.

    DeprecationWarnings were already emitted if :class:`typing.ByteString` was subclassed or used as the second argument to :func:`isinstance` or :func:`issubclass`, but warnings were not previously emitted if it was merely imported or accessed from the :mod:`!typing` module.

  • Deprecated :func:`!typing.no_type_check_decorator` has been removed. (Contributed by Nikita Sobolev in :gh:`133601`.)

wave

zipimport

Deprecated

New deprecations

C API changes

New features

Changed C APIs

Porting to Python 3.15

  • Private functions promoted to public C APIs:

    The pythoncapi-compat project can be used to get most of these new functions on Python 3.14 and older.

Removed C APIs

The following functions are removed in favor of :c:func:`PyConfig_Get`. The pythoncapi-compat project can be used to get :c:func:`!PyConfig_Get` on Python 3.13 and older.

Deprecated C APIs

Build changes

Porting to Python 3.15

This section lists previously described changes and other bugfixes that may require changes to your code.