Skip to content
Open
Show file tree
Hide file tree
Changes from 70 commits
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
d3d6e64
Resolve first positional param, required to be annotated
johnslavik Jan 6, 2026
6ea2a4a
Special-case strings for forward refs similarly to typing
johnslavik Jan 6, 2026
0a39278
Rename `ref_or_type` to `ref_or_typeform`
johnslavik Jan 6, 2026
096fc3b
Add comment
johnslavik Jan 6, 2026
c8a5cdc
Shorten error message string line
johnslavik Jan 6, 2026
e1cde59
Adjust formatting to functools style
johnslavik Jan 6, 2026
6bc698b
Normalize `None` to a type, strip annotations
johnslavik Jan 6, 2026
4ef7c7c
Rename `ref_or_typeform` to `fwdref_or_typeform`
johnslavik Jan 6, 2026
004c852
Rename `skip_first` to `skip_first_param`
johnslavik Jan 6, 2026
9bc1436
Add news entry
johnslavik Jan 6, 2026
3115fd7
Add GH-130827 test
johnslavik Jan 3, 2026
f6c102f
Fix test
johnslavik Jan 6, 2026
7968570
Remove the `get_annotations` dance for now
johnslavik Jan 6, 2026
a808a1e
Fix incorrect `regster()` calls in `TestSingleDispatch.test_method_si…
johnslavik Jan 7, 2026
69b9978
Fix string signatures accordingly
johnslavik Jan 7, 2026
ebdb68d
Raise exception if positional argument not found
johnslavik Jan 7, 2026
8d86f9e
Break the exception chain
johnslavik Jan 7, 2026
82616f9
Support all callables
johnslavik Jan 7, 2026
c9a1f1a
Clarify comment
johnslavik Jan 7, 2026
e878207
Fiat lux, inline validation
johnslavik Jan 7, 2026
d3240a3
Add more test cases (mainly wrappers)
johnslavik Jan 7, 2026
9240b0d
More comments!
johnslavik Jan 7, 2026
f6ccb97
Less history pollution
johnslavik Jan 7, 2026
6642321
Document `_get_positional_param`
johnslavik Jan 7, 2026
0eaaa5b
Better comments!
johnslavik Jan 7, 2026
345b7e9
Shorten a comment
johnslavik Jan 7, 2026
8a46f3f
Rephrase the fallback path comment
johnslavik Jan 7, 2026
16f83ee
Improve the error message when missing an annotation
johnslavik Jan 7, 2026
552daaf
Correct the docstring
johnslavik Jan 7, 2026
113cc29
Rephrase the documentation again
johnslavik Jan 7, 2026
7c1bcea
Rename the function to `_get_dispatch_param`
johnslavik Jan 7, 2026
444425c
Rewrite the news entry using precise language
johnslavik Jan 7, 2026
9b26fb1
Add a test for positional-only parameter
johnslavik Jan 7, 2026
17dfb36
Add a mixed parameter types test case
johnslavik Jan 7, 2026
fbce76d
Do not break exception chain unnecessarily
johnslavik Jan 7, 2026
44b8bba
Improve the docstring
johnslavik Jan 7, 2026
ec01821
Add precedent case for GH-84644
johnslavik Jan 7, 2026
57faa34
Fix GH-84644 test
johnslavik Jan 7, 2026
e4fb514
Reword the documentation of `_get_dispatch_param`
johnslavik Jan 7, 2026
57965a9
Merge GH-130827 test into `test_method_type_ann_register`
johnslavik Jan 7, 2026
eadc38f
Add case this PR broke -- registering bound methods
johnslavik Jan 7, 2026
682c41e
Add bound methods to slow path
johnslavik Jan 7, 2026
c406755
Optimize instance checks in the fast path
johnslavik Jan 7, 2026
e238e6a
Use a match statement instead of a for loop
johnslavik Jan 7, 2026
1e61429
Rewrite to a try-except
johnslavik Jan 7, 2026
19458fc
Improve comment
johnslavik Jan 7, 2026
6390a82
Add more bound method tests
johnslavik Jan 7, 2026
3e33040
Reuse one instance of test class
johnslavik Jan 7, 2026
c50d344
Test instance validity in bound method tests
johnslavik Jan 7, 2026
32910f3
Tests and fixes for staticmethod
johnslavik Jan 7, 2026
4283fba
Add more tests for classmethod
johnslavik Jan 7, 2026
17b5088
Disambiguate a comment
johnslavik Jan 8, 2026
cdb7cca
Always respect descriptors, fallback to assumptions on function-like …
johnslavik Jan 8, 2026
62088c7
Add more comments
johnslavik Jan 8, 2026
c497857
Specialcase bound methods in singledispatchmethods
johnslavik Jan 8, 2026
0f75d98
Finalize the logic
johnslavik Jan 8, 2026
8350e71
Crystalize the decision tree
johnslavik Jan 8, 2026
50c0e64
Fix comment
johnslavik Jan 8, 2026
052c2fd
Better comments
johnslavik Jan 8, 2026
30994eb
Fiat lux
johnslavik Jan 8, 2026
3edad44
Rename function to `_get_singledispatch_annotated_param`
johnslavik Jan 8, 2026
fbc205e
Disambiguate comment
johnslavik Jan 8, 2026
b691969
More comments
johnslavik Jan 8, 2026
0859bc0
Add more missing tests
johnslavik Jan 8, 2026
7ac8275
Cast the `idx` to an integer explicitly
johnslavik Jan 8, 2026
fbe00f8
Check param kinds by name (code review)
johnslavik Jan 8, 2026
7ada2b0
Minime the try-except
johnslavik Jan 8, 2026
ac2f5a2
Remove all new tests
johnslavik Jan 8, 2026
ba46e43
Add previously failing tests only
johnslavik Jan 8, 2026
3995e79
Merge branch 'main' into fix-singledispatch-annotation-parsing
johnslavik Jan 12, 2026
48d1bde
Use imperative form in the docstring
johnslavik Jan 19, 2026
fc5df46
Raise in `_get_singledispatch_annotated_param`
johnslavik Jan 20, 2026
7fcf4d5
Rename the ugly `_inside_dispatchmethod` to `__role__`
johnslavik Jan 20, 2026
f7ec61d
Hide private params from `.register` using `__text_signature__`
johnslavik Jan 20, 2026
946ccb8
Remove comments that are sorta obvious
johnslavik Jan 20, 2026
2d1180a
Make the private helper tighter
johnslavik Jan 20, 2026
cd10e91
Condense fast path comment
johnslavik Jan 20, 2026
0a622fb
Bring back comments but make them actually helpful
johnslavik Jan 20, 2026
9b2d20d
Another comment improvement
johnslavik Jan 20, 2026
9191a22
Fix incorrect comment
johnslavik Jan 20, 2026
ef74667
Restore old tests
johnslavik Jan 20, 2026
b55de76
Rework registration tests
johnslavik Jan 20, 2026
9977251
Use stars for referring to param names
johnslavik Jan 20, 2026
d282c9a
Use `__role__` name only in the `register()` signature
johnslavik Jan 20, 2026
4c43a9c
Denote the issues covered
johnslavik Jan 20, 2026
3b5a410
Add regrtest from @pR0Ps
pR0Ps Jan 20, 2026
f6ce233
Somewhat better comment
johnslavik Jan 20, 2026
97a8515
Merge branch 'main' into fix-singledispatch-annotation-parsing
johnslavik Jan 22, 2026
62ea333
Apply suggestion from @johnslavik
johnslavik Jan 22, 2026
f0097c4
Final optional refinements
johnslavik Feb 8, 2026
4e6a003
Bring back `role` and `__role__`
johnslavik Feb 11, 2026
2d70e22
Fix code line length in `_get_singledispatch_annotated_param`
johnslavik Mar 1, 2026
c716b57
Rename `__role__` to `_role` and make it kw-only
johnslavik Mar 1, 2026
a28e9af
Fix missing param default in `__text_signature__`
johnslavik Mar 1, 2026
cfb9194
Change wording in news entry patch attribution
johnslavik Mar 1, 2026
b03891f
Rephrase the entire news entry
johnslavik Mar 1, 2026
4c8e818
Finish up news entry corrections
johnslavik Mar 1, 2026
7fa9315
Apply more consistent formatting
johnslavik Mar 1, 2026
cee055a
...Actually readable formatting
johnslavik Mar 1, 2026
4db216f
Merge main to avoid conflict in the "What's New" entry
johnslavik Mar 1, 2026
cc2bcc7
Rephrase the news entry again
johnslavik Mar 1, 2026
371bff5
Add a what's new entry
johnslavik Mar 1, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 64 additions & 4 deletions Lib/functools.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
# import weakref # Deferred to single_dispatch()
from operator import itemgetter
from reprlib import recursive_repr
from types import GenericAlias, MethodType, MappingProxyType, UnionType
from types import FunctionType, GenericAlias, MethodType, MappingProxyType, UnionType
from _thread import RLock

################################################################################
Expand Down Expand Up @@ -888,6 +888,48 @@ def _find_impl(cls, registry):
match = t
return registry.get(match)

def _get_singledispatch_annotated_param(func, *, _inside_dispatchmethod=False):
"""Finds the first positional and user-specified parameter in a callable
Comment thread
johnslavik marked this conversation as resolved.
Outdated
or descriptor.

Used by singledispatch for registration by type annotation of the parameter.
"""
# Pick the first parameter if function had @staticmethod.
if isinstance(func, staticmethod):
idx = 0
func = func.__func__
# Pick the second parameter if function had @classmethod or is a bound method.
elif isinstance(func, (classmethod, MethodType)):
idx = 1
func = func.__func__
# If it is a regular function:
# Pick the first parameter if registering via singledispatch.
# Pick the second parameter if registering via singledispatchmethod.
else:
idx = int(_inside_dispatchmethod)

# If it is a simple function, try to read from the code object fast.
if isinstance(func, FunctionType) and not hasattr(func, "__wrapped__"):
# Emulate inspect._signature_from_function to get the desired parameter.
func_code = func.__code__
try:
return func_code.co_varnames[:func_code.co_argcount][idx]
except IndexError:
pass

# Fall back to inspect.signature (slower, but complete).
import inspect
params = list(inspect.signature(func).parameters.values())
try:
param = params[idx]
except IndexError:
Comment thread
johnslavik marked this conversation as resolved.
pass
else:
# Allow variadic positional "(*args)" parameters for backward compatibility.
if param.kind not in (inspect.Parameter.KEYWORD_ONLY, inspect.Parameter.VAR_KEYWORD):
return param.name
return None

def singledispatch(func):
"""Single-dispatch generic function decorator.

Expand Down Expand Up @@ -935,7 +977,7 @@ def _is_valid_dispatch_type(cls):
return (isinstance(cls, UnionType) and
all(isinstance(arg, type) for arg in cls.__args__))

def register(cls, func=None):
def register(cls, func=None, _inside_dispatchmethod=False):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider changing this from a boolean to a scope parameter, where scope is one of function or method. Such a form would be more descriptive and flexible for the possibility of additional scopes.

Copy link
Copy Markdown
Member Author

@johnslavik johnslavik Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this semantics.

scope sounds quite good as a name, but I reckon that I'd call it role or purpose, because any callable can be a standalone function or a method simply depending on its purpose in the code.

Copy link
Copy Markdown
Member Author

@johnslavik johnslavik Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OTOH, maybe we could change the parameter name to _skip_first_arg. That's what it's essentially doing.

I don't think this is likely there would exist any other scope / role / purpose than function (0) and method (1).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like role="function" | "method"

Copy link
Copy Markdown
Member Author

@johnslavik johnslavik Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JelleZijlstra role as in _role or __role__? A new public parameter would be a feature which would block us from backporting this, no?

Copy link
Copy Markdown
Member Author

@johnslavik johnslavik Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we intend _role or __role__ -- I was told about the concern that a private param like this can sneak into help() text. One of the ways to overcome it is to assign a custom __text_signature__. We could also encapsulate the underlying logic into a private function and expose public wrappers free of internal flags, but I don't like it because it pollutes history and makes the code bigger.

I went with __role__ and __text_signature__. Lmk if you'd like an iteration on that.

"""generic_func.register(cls, func) -> func

Registers a new implementation for the given *cls* on a *generic_func*.
Expand All @@ -960,10 +1002,28 @@ def register(cls, func=None):
)
func = cls

argname = _get_singledispatch_annotated_param(
func, _inside_dispatchmethod=_inside_dispatchmethod)
if argname is None:
Comment thread
johnslavik marked this conversation as resolved.
Outdated
raise TypeError(
Comment thread
johnslavik marked this conversation as resolved.
Outdated
f"Invalid first argument to `register()`: {func!r} "
f"does not accept positional arguments."
)

# only import typing if annotation parsing is necessary
from typing import get_type_hints
from annotationlib import Format, ForwardRef
argname, cls = next(iter(get_type_hints(func, format=Format.FORWARDREF).items()))
annotations = get_type_hints(func, format=Format.FORWARDREF)

try:
cls = annotations[argname]
except KeyError:
raise TypeError(
f"Invalid first argument to `register()`: {func!r}. "
"Use either `@register(some_class)` or add a type "
f"annotation to parameter {argname!r} of your callable."
) from None

if not _is_valid_dispatch_type(cls):
if isinstance(cls, UnionType):
raise TypeError(
Expand Down Expand Up @@ -1027,7 +1087,7 @@ def register(self, cls, method=None):

Registers a new implementation for the given *cls* on a *generic_method*.
"""
return self.dispatcher.register(cls, func=method)
return self.dispatcher.register(cls, func=method, _inside_dispatchmethod=True)

def __get__(self, obj, cls=None):
return _singledispatchmethod_get(self, obj, cls)
Expand Down
65 changes: 64 additions & 1 deletion Lib/test/test_functools.py
Original file line number Diff line number Diff line change
Expand Up @@ -2905,13 +2905,34 @@ def t(self, arg):
def _(self, arg: int):
return "int"
@t.register
def _(self, arg: str):
def _(self, arg: complex, /):
return "complex"
@t.register
def _(self, /, arg: str):
return "str"
# See GH-130827.
def wrapped1(self: typing.Self, arg: bytes):
return "bytes"
@t.register
@functools.wraps(wrapped1)
def wrapper1(self, *args, **kwargs):
return self.wrapped1(*args, **kwargs)

def wrapped2(self, arg: bytearray) -> str:
return "bytearray"
@t.register
@functools.wraps(wrapped2)
def wrapper2(self, *args: typing.Any, **kwargs: typing.Any):
return self.wrapped2(*args, **kwargs)

a = A()

self.assertEqual(a.t(0), "int")
self.assertEqual(a.t(0j), "complex")
self.assertEqual(a.t(''), "str")
self.assertEqual(a.t(0.0), "base")
self.assertEqual(a.t(b''), "bytes")
self.assertEqual(a.t(bytearray()), "bytearray")

def test_staticmethod_type_ann_register(self):
class A:
Expand Down Expand Up @@ -3172,12 +3193,27 @@ def test_invalid_registrations(self):
@functools.singledispatch
def i(arg):
return "base"
with self.assertRaises(TypeError) as exc:
@i.register
def _() -> None:
return "My function doesn't take arguments"
self.assertStartsWith(str(exc.exception), msg_prefix)
self.assertEndsWith(str(exc.exception), "does not accept positional arguments.")

with self.assertRaises(TypeError) as exc:
@i.register
def _(*, foo: str) -> None:
return "My function takes keyword-only arguments"
self.assertStartsWith(str(exc.exception), msg_prefix)
self.assertEndsWith(str(exc.exception), "does not accept positional arguments.")
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate the tests, but I feel like this change possible deserves its own test cases and not simply expansions of the existing test cases. I haven't looked deeply, so my instincts may be wrong here, but do consider creating independent tests where feasible and not too intrusive on the existing tests.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I'll keep this open. I'll appreciate additional feedback about the preferred way forward.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests seem similar enough to the other cases in test_invalid_registrations that it makes sense to keep them together.

On the other hand the function is big enough that it might be nice to split it up. A nice split could be to have one test for cases where we can't get a type at all (i.e., there are no annotations in the place where we look for them) and another test case for cases where we do find an annotation, but singledispatch doesn't know what to do with it.

Copy link
Copy Markdown
Member Author

@johnslavik johnslavik Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this. I'll do that. Thanks, I haven't thought about it.

Copy link
Copy Markdown
Member Author

@johnslavik johnslavik Jan 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JelleZijlstra, I removed all tests (including additional test_invalid_registrations cases) in ef74667, rearranged the existing cases in test_invalid_registrations to separate the concerns you mentioned, and added brand new separate cases (in b55de76). Does this more or less match what you had in mind?

another test case for cases where we do find an annotation, but singledispatch doesn't know what to do with it.

I found these to be already covered after splitting, so I didn't add this aspect to my new cases (which reimplement the removed ones in a more structured way).


with self.assertRaises(TypeError) as exc:
@i.register(42)
def _(arg):
return "I annotated with a non-type"
Comment thread
johnslavik marked this conversation as resolved.
Outdated
self.assertStartsWith(str(exc.exception), msg_prefix + "42")
self.assertEndsWith(str(exc.exception), msg_suffix)

with self.assertRaises(TypeError) as exc:
@i.register
def _(arg):
Expand All @@ -3187,6 +3223,33 @@ def _(arg):
)
self.assertEndsWith(str(exc.exception), msg_suffix)

with self.assertRaises(TypeError) as exc:
@i.register
def _(arg, extra: int):
return "I did not annotate the right param"
self.assertStartsWith(str(exc.exception), msg_prefix +
"<function TestSingleDispatch.test_invalid_registrations.<locals>._"
)
self.assertEndsWith(str(exc.exception),
"Use either `@register(some_class)` or add a type annotation "
f"to parameter 'arg' of your callable.")

with self.assertRaises(TypeError) as exc:
# See GH-84644.

@functools.singledispatch
def func(arg):...

@func.register
def _int(arg) -> int:...

self.assertStartsWith(str(exc.exception), msg_prefix +
"<function TestSingleDispatch.test_invalid_registrations.<locals>._int"
)
self.assertEndsWith(str(exc.exception),
"Use either `@register(some_class)` or add a type annotation "
f"to parameter 'arg' of your callable.")

with self.assertRaises(TypeError) as exc:
@i.register
def _(arg: typing.Iterable[str]):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
:func:`functools.singledispatch` and :func:`functools.singledispatchmethod`
now require callables to be correctly annotated if registering without a type explicitly
specified in the decorator. The first user-specified positional parameter of a callable
must always be annotated. Before, a callable could be registered based on its return type
Comment thread
johnslavik marked this conversation as resolved.
Outdated
annotation or based on an irrelevant parameter type annotation. Contributed by Bartosz Sławecki.
Comment thread
johnslavik marked this conversation as resolved.
Outdated
Comment thread
johnslavik marked this conversation as resolved.
Outdated
Loading