Initial list strategy work (#540)

* Initial list strategy work

* Black reformat

* More docs

* Changelog

* More sets for cols

* More history

* Add test for better recursive structuring

* Improve set handling on 3.8

* Docs

* Docs

Author: Tinche

HISTORY.md

@@ -27,6 +27,8 @@ Our backwards-compatibility policy can be found [here](https://github.com/python
 can now be used as decorators and have gained new features.
   See [here](https://catt.rs/en/latest/customizing.html#use-as-decorators) and [here](https://catt.rs/en/latest/customizing.html#id1) for more details.
   ([#487](https://github.com/python-attrs/cattrs/pull/487))
+- Introduce and [document](https://catt.rs/en/latest/customizing.html#customizing-collections) the {mod}`cattrs.cols` module for better collection customizations.
+  ([#504](https://github.com/python-attrs/cattrs/issues/504) [#540](https://github.com/python-attrs/cattrs/pull/540))
 - Introduce the [_msgspec_](https://jcristharif.com/msgspec/) {mod}`preconf converter <cattrs.preconf.msgspec>`.
   Only JSON is supported for now, with other formats supported by _msgspec_ to come later.
   ([#481](https://github.com/python-attrs/cattrs/pull/481))
@@ -46,8 +48,10 @@ can now be used as decorators and have gained new features.
   ([#481](https://github.com/python-attrs/cattrs/pull/481))
 - The {class}`orjson preconf converter <cattrs.preconf.orjson.OrjsonConverter>` now passes through dates and datetimes to orjson while unstructuring, greatly improving speed.
   ([#463](https://github.com/python-attrs/cattrs/pull/463))
-- `cattrs.gen` generators now attach metadata to the generated functions, making them introspectable.
+- {mod}`cattrs.gen` generators now attach metadata to the generated functions, making them introspectable.
   ([#472](https://github.com/python-attrs/cattrs/pull/472))
+- Structure hook factories in {mod}`cattrs.gen` now handle recursive classes better.
+  ([#540](https://github.com/python-attrs/cattrs/pull/540))
 - The [tagged union strategy](https://catt.rs/en/stable/strategies.html#tagged-unions-strategy) now leaves the tags in the payload unless `forbid_extra_keys` is set.
   ([#533](https://github.com/python-attrs/cattrs/issues/533) [#534](https://github.com/python-attrs/cattrs/pull/534))
 - More robust support for `Annotated` and `NotRequired` in TypedDicts.

docs/basics.md

@@ -14,7 +14,7 @@ To create a private converter, instantiate a {class}`cattrs.Converter`. Converte
 
 The two main methods, {meth}`structure <cattrs.BaseConverter.structure>` and {meth}`unstructure <cattrs.BaseConverter.unstructure>`, are used to convert between _structured_ and _unstructured_ data.
 
-```python
+```{doctest} basics
 >>> from cattrs import structure, unstructure
 >>> from attrs import define
 
@@ -23,40 +23,39 @@ The two main methods, {meth}`structure <cattrs.BaseConverter.structure>` and {me
 ...    a: int
 
 >>> unstructure(Model(1))
-{"a": 1}
+{'a': 1}
 >>> structure({"a": 1}, Model)
 Model(a=1)
 ```
 
 _cattrs_ comes with a rich library of un/structuring hooks by default but it excels at composing custom hooks with built-in ones.
 
 The simplest approach to customization is writing a new hook from scratch.
-For example, we can write our own hook for the `int` class.
+For example, we can write our own hook for the `int` class and register it to a converter.
 
-```python
->>> def int_hook(value, type):
+```{doctest} basics
+>>> from cattrs import Converter
+
+>>> converter = Converter()
+
+>>> @converter.register_structure_hook
+... def int_hook(value, type) -> int:
 ...     if not isinstance(value, int):
 ...         raise ValueError('not an int!')
 ...     return value
 ```
 
-We can then register this hook to a converter and any other hook converting an `int` will use it.
-
-```python
->>> from cattrs import Converter
-
->>> converter = Converter()
->>> converter.register_structure_hook(int, int_hook)
-```
+Now, any other hook converting an `int` will use it.
 
-Another approach to customization is wrapping an existing hook with your own function.
+Another approach to customization is wrapping (composing) an existing hook with your own function.
 A base hook can be obtained from a converter and then be subjected to the very rich machinery of function composition that Python offers.
 
 
-```python
+```{doctest} basics
 >>> base_hook = converter.get_structure_hook(Model)
 
->>> def my_model_hook(value, type):
+>>> @converter.register_structure_hook
+... def my_model_hook(value, type) -> Model:
 ...     # Apply any preprocessing to the value.
 ...     result = base_hook(value, type)
 ...     # Apply any postprocessing to the model.
@@ -65,13 +64,6 @@ A base hook can be obtained from a converter and then be subjected to the very r
 
 (`cattrs.structure({}, Model)` is equivalent to `cattrs.get_structure_hook(Model)({}, Model)`.)
 
-This new hook can be used directly or registered to a converter (the original instance, or a different one):
-
-```python
->>> converter.register_structure_hook(Model, my_model_hook)
-```
-
-
 Now if we use this hook to structure a `Model`, through ✨the magic of function composition✨ that hook will use our old `int_hook`.
 
 ```python

docs/cattrs.rst

@@ -19,6 +19,14 @@ Subpackages
 Submodules
 ----------
 
+cattrs.cols module
+------------------
+
+.. automodule:: cattrs.cols
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 cattrs.disambiguators module
 ----------------------------
 

docs/conf.py

@@ -155,6 +155,76 @@ Here's an example of using an unstructure hook factory to handle unstructuring [
 [1, 2]
 ```
 
+## Customizing Collections
+
+The {mod}`cattrs.cols` module contains predicates and hook factories useful for customizing collection handling.
+These hook factories can be wrapped to apply complex customizations.
+
+Available predicates are:
+
+* {meth}`is_any_set <cattrs.cols.is_any_set>`
+* {meth}`is_frozenset <cattrs.cols.is_frozenset>`
+* {meth}`is_set <cattrs.cols.is_set>`
+* {meth}`is_sequence <cattrs.cols.is_sequence>`
+* {meth}`is_namedtuple <cattrs.cols.is_namedtuple>`
+
+````{tip}
+These predicates aren't _cattrs_-specific and may be useful in other contexts.
+```{doctest} predicates
+>>> from cattrs.cols import is_sequence
+
+>>> is_sequence(list[str])
+True
+```
+````
+
+
+Available hook factories are:
+
+* {meth}`iterable_unstructure_factory <cattrs.cols.iterable_unstructure_factory>`
+* {meth}`list_structure_factory <cattrs.cols.list_structure_factory>`
+* {meth}`namedtuple_structure_factory <cattrs.cols.namedtuple_structure_factory>`
+* {meth}`namedtuple_unstructure_factory <cattrs.cols.namedtuple_unstructure_factory>`
+
+Additional predicates and hook factories will be added as requested.
+
+For example, by default sequences are structured from any iterable into lists.
+This may be too lax, and additional validation may be applied by wrapping the default list structuring hook factory.
+
+```{testcode} list-customization
+from cattrs.cols import is_sequence, list_structure_factory
+
+c = Converter()
+
+@c.register_structure_hook_factory(is_sequence)
+def strict_list_hook_factory(type, converter):
+
+    # First, we generate the default hook...
+    list_hook = list_structure_factory(type, converter)
+
+    # Then, we wrap it with a function of our own...
+    def strict_list_hook(value, type):
+        if not isinstance(value, list):
+            raise ValueError("Not a list!")
+        return list_hook(value, type)
+
+    # And finally, we return our own composite hook.
+    return strict_list_hook
+```
+
+Now, all sequence structuring will be stricter:
+
+```{doctest} list-customization
+>>> c.structure({"a", "b", "c"}, list[str])
+Traceback (most recent call last):
+    ...
+ValueError: Not a list!
+```
+
+```{versionadded} 24.1.0
+
+```
+
 ## Using `cattrs.gen` Generators
 
 The {mod}`cattrs.gen` module allows for generating and compiling specialized hooks for unstructuring _attrs_ classes, dataclasses and typed dicts.

docs/customizing.md

@@ -23,6 +23,10 @@ The new copy may be changed through the `copy` arguments, but will retain all ma
 This feature is supported for Python 3.9 and later.
 ```
 
+```{tip}
+See [](customizing.md#customizing-collections) for a more modern and more powerful way of customizing collection handling.
+```
+
 Overriding collection unstructuring in a generic way can be a very useful feature.
 A common example is using a JSON library that doesn't support sets, but expects lists and tuples instead.
 

docs/indepth.md

@@ -37,6 +37,7 @@ caption: Dev Guide
 history
 benchmarking
 contributing
+modindex
 ```
 
 ```{include} ../README.md

docs/index.md

@@ -376,6 +376,7 @@ This strategy has been preapplied to the following preconfigured converters:
 - {py:class}`Cbor2Converter <cattrs.preconf.cbor2.Cbor2Converter>`
 - {py:class}`JsonConverter <cattrs.preconf.json.JsonConverter>`
 - {py:class}`MsgpackConverter <cattrs.preconf.msgpack.MsgpackConverter>`
+- {py:class}`MsgspecJsonConverter <cattrs.preconf.msgspec.MsgspecJsonConverter>`
 - {py:class}`OrjsonConverter <cattrs.preconf.orjson.OrjsonConverter>`
 - {py:class}`PyyamlConverter <cattrs.preconf.pyyaml.PyyamlConverter>`
 - {py:class}`TomlkitConverter <cattrs.preconf.tomlkit.TomlkitConverter>`

docs/strategies.md

@@ -332,6 +332,11 @@ def get_notrequired_base(type) -> "Union[Any, Literal[NOTHING]]":
         return NOTHING
 
     def is_sequence(type: Any) -> bool:
+        """A predicate function for sequences.
+
+        Matches lists, sequences, mutable sequences, deques and homogenous
+        tuples.
+        """
         origin = getattr(type, "__origin__", None)
         return (
             type
@@ -366,7 +371,11 @@ def is_deque(type):
             or (getattr(type, "__origin__", None) is deque)
         )
 
-    def is_mutable_set(type):
+    def is_mutable_set(type: Any) -> bool:
+        """A predicate function for (mutable) sets.
+
+        Matches built-in sets and sets from the typing module.
+        """
         return (
             type in (TypingSet, TypingMutableSet, set)
             or (
@@ -376,7 +385,11 @@ def is_mutable_set(type):
             or (getattr(type, "__origin__", None) in (set, AbcMutableSet, AbcSet))
         )
 
-    def is_frozenset(type):
+    def is_frozenset(type: Any) -> bool:
+        """A predicate function for frozensets.
+
+        Matches built-in frozensets and frozensets from the typing module.
+        """
         return (
             type in (FrozenSet, frozenset)
             or (
@@ -491,9 +504,10 @@ def is_deque(type: Any) -> bool:
             or type.__origin__ is deque
         )
 
-    def is_mutable_set(type):
-        return type is set or (
-            type.__class__ is _GenericAlias and is_subclass(type.__origin__, MutableSet)
+    def is_mutable_set(type) -> bool:
+        return type in (set, TypingAbstractSet) or (
+            type.__class__ is _GenericAlias
+            and is_subclass(type.__origin__, (MutableSet, TypingAbstractSet))
         )
 
     def is_frozenset(type):