union passthrough (#403)

* union passthrough, first pass

* 3.7 has no types.NoneType

* No literals on 3.7

* More native unions work

* Fix maybe?

* Fixety fix

* docs in progress

* More native unions

* More union passthrough tests, docs

* Fix test

* Add test and more docs

* Pyyaml work, docs

* Skip testing literals on 3.7

* No literals on 3.7

* Test with Literal from typing

* Fix test

* Fix native unions

* Tweak tests on 3.8

* NewType tests

* More NewType work

* More NewTypes work

* Fix test

* Final tweaks

Author: Tinche

HISTORY.md

@@ -10,6 +10,7 @@
   ([#410](https://github.com/python-attrs/cattrs/issues/410) [#411](https://github.com/python-attrs/cattrs/pull/411))
 - Introduce the `use_class_methods` strategy. Learn more [here](https://catt.rs/en/latest/strategies.html#using-class-specific-structure-and-unstructure-methods).
   ([#405](https://github.com/python-attrs/cattrs/pull/405))
+- Implement the `union passthrough` strategy, enabling much richer union handling for preconfigured converters. [Learn more here](https://catt.rs/en/stable/strategies.html#union-passthrough).
 - The `omit` parameter of {py:func}`cattrs.override` is now of type `bool | None` (from `bool`).
   `None` is the new default and means to apply default _cattrs_ handling to the attribute, which is to omit the attribute if it's marked as `init=False`, and keep it otherwise.
 - Fix {py:func}`format_exception() <cattrs.v.format_exception>` parameter working for recursive calls to {py:func}`transform_error <cattrs.transform_error>`.
@@ -40,6 +41,8 @@
   ([#418](https://github.com/python-attrs/cattrs/issues/418))
 - Add support for `date` to preconfigured converters.
   ([#420](https://github.com/python-attrs/cattrs/pull/420))
+- Add support for `datetime.date`s to the PyYAML preconfigured converter.
+  ([#393](https://github.com/python-attrs/cattrs/issues/393))
 
 ## 23.1.2 (2023-06-02)
 

docs/_static/custom.css

@@ -101,6 +101,7 @@ span:target ~ h4:first-of-type,
 span:target ~ h5:first-of-type,
 span:target ~ h6:first-of-type {
   text-decoration: underline dashed;
+  text-decoration-thickness: 1px;
 }
 
 div.article-container > article {

docs/strategies.md

@@ -323,3 +323,62 @@ Nested(m=MyClass(a=43))
 ```{versionadded} 23.2.0
 
 ```
+
+## Union Passthrough
+
+_Found at {py:func}`cattrs.strategies.configure_union_passthrough`._
+
+The _union passthrough_ strategy enables a {py:class}`Converter <cattrs.BaseConverter>` to structure unions and subunions of given types.
+
+A very common use case for _cattrs_ is processing data created by other serialization libraries, such as _JSON_ or _msgpack_.
+These libraries are able to directly produce values of unions inherent to the format.
+For example, every JSON library can differentiate between numbers, booleans, strings and null values since these values are represented differently in the wire format.
+This strategy enables _cattrs_ to offload the creation of these values to an underlying library and just validate the final value.
+So, _cattrs_ preconfigured JSON converters can handle the following type:
+
+- `bool | int | float | str | None`
+
+Continuing the JSON example, this strategy also enables structuring subsets of unions of these values.
+Accordingly, here are some examples of subset unions that are also supported:
+
+- `bool | int`
+- `int | str`
+- `int | float | str`
+
+The strategy also supports types including one or more [Literals](https://mypy.readthedocs.io/en/stable/literal_types.html#literal-types) of supported types. For example:
+
+- `Literal["admin", "user"] | int`
+- `Literal[True] | str | int | float`
+
+The strategy also supports [NewTypes](https://mypy.readthedocs.io/en/stable/more_types.html#newtypes) of these types. For example:
+
+```python
+>>> from typing import NewType
+
+>>> UserId = NewType("UserId", int)
+
+>>> converter.loads("12", UserId)
+12
+```
+
+Unions containing unsupported types can be handled if at least one union type is supported by the strategy; the supported union types will be checked before the rest (referred to as the _spillover_) is handed over to the converter again.
+
+For example, if `A` and `B` are arbitrary _attrs_ classes, the union `Literal[10] | A | B` cannot be handled directly by a JSON converter.
+However, the strategy will check if the value being structured matches `Literal[10]` (because this type _is_ supported) and, if not, will pass it back to the converter to be structured as `A | B` (where a different strategy can handle it).
+
+The strategy is designed to run in _O(1)_ at structure time; it doesn't depend on the size of the union and the ordering of union members.
+
+This strategy has been preapplied to the following preconfigured converters:
+
+- {py:class}`BsonConverter <cattrs.preconf.bson.BsonConverter>`
+- {py:class}`Cbor2Converter <cattrs.preconf.cbor2.Cbor2Converter>`
+- {py:class}`JsonConverter <cattrs.preconf.json.JsonConverter>`
+- {py:class}`MsgpackConverter <cattrs.preconf.msgpack.MsgpackConverter>`
+- {py:class}`OrjsonConverter <cattrs.preconf.orjson.OrjsonConverter>`
+- {py:class}`PyyamlConverter <cattrs.preconf.pyyaml.PyyamlConverter>`
+- {py:class}`TomlkitConverter <cattrs.preconf.tomlkit.TomlkitConverter>`
+- {py:class}`UjsonConverter <cattrs.preconf.ujson.UjsonConverter>`
+
+```{versionadded} 23.2.0
+
+```

pdm.lock

@@ -8,6 +8,32 @@ known_first_party = ["cattr"]
 [tool.hatch.build.targets.wheel]
 packages = ["src/cattr", "src/cattrs"]
 
+
+[tool.pdm.dev-dependencies]
+lint = [
+    "isort>=5.11.5",
+    "black>=23.3.0",
+    "ruff>=0.0.277",
+]
+test = [
+    "hypothesis>=6.79.4",
+    "pytest>=7.4.0",
+    "pytest-benchmark>=4.0.0",
+    "immutables>=0.19",
+    "typing-extensions>=4.7.1",
+    "coverage>=7.2.7",
+]
+docs = [
+    "sphinx>=5.3.0",
+    "furo>=2023.3.27",
+    "sphinx-copybutton>=0.5.2",
+    "myst-parser>=1.0.0",
+    "pendulum>=2.1.2",
+]
+bench = [
+    "pyperf>=2.6.1",
+]
+
 [build-system]
 requires = ["hatchling"]
 build-backend = "hatchling.build"
@@ -62,28 +88,6 @@ bson = [
     "pymongo>=4.4.0",
 ]
 
-[tool.pdm.dev-dependencies]
-lint = [
-    "isort>=5.11.5",
-    "black>=23.3.0",
-    "ruff>=0.0.277",
-]
-test = [
-    "hypothesis>=6.79.4",
-    "pytest>=7.4.0",
-    "pytest-benchmark>=4.0.0",
-    "immutables>=0.19",
-    "typing-extensions>=4.7.1",
-    "coverage>=7.2.7",
-]
-docs = [
-    "sphinx>=5.3.0",
-    "furo>=2023.3.27",
-    "sphinx-copybutton>=0.5.2",
-    "myst-parser>=1.0.0",
-    "pendulum>=2.1.2",
-]
-
 [tool.pytest.ini_options]
 addopts = "-l --benchmark-sort=fullname --benchmark-warmup=true --benchmark-warmup-iterations=5  --benchmark-group-by=fullname"
 
@@ -111,6 +115,7 @@ select = [
     "B",   # flake8-bugbear
     "C4",  # flake8-comprehensions
     "T10", # flake8-debugger
+    "T20", # flake8-print
     "ISC", # flake8-implicit-str-concat
     "RET", # flake8-return
     "SIM", # flake8-simplify

pyproject.toml

@@ -1,14 +1,15 @@
 """Preconfigured converters for bson."""
 from base64 import b85decode, b85encode
-from datetime import datetime, date
-from typing import Any, Type, TypeVar
+from datetime import date, datetime
+from typing import Any, Type, TypeVar, Union
 
-from bson import DEFAULT_CODEC_OPTIONS, CodecOptions, ObjectId, decode, encode
+from bson import DEFAULT_CODEC_OPTIONS, CodecOptions, Int64, ObjectId, decode, encode
 
 from cattrs._compat import AbstractSet, is_mapping
 from cattrs.gen import make_mapping_structure_fn
 
 from ..converters import BaseConverter, Converter
+from ..strategies import configure_union_passthrough
 from . import validate_datetime
 
 T = TypeVar("T")
@@ -83,6 +84,9 @@ def gen_structure_mapping(cl: Any):
     )
 
     converter.register_structure_hook(ObjectId, lambda v, _: ObjectId(v))
+    configure_union_passthrough(
+        Union[str, bool, int, float, None, bytes, datetime, ObjectId, Int64], converter
+    )
 
     # datetime inherits from date, so identity unstructure hook used
     # here to prevent the date unstructure hook running.

src/cattrs/preconf/bson.py

@@ -1,12 +1,13 @@
 """Preconfigured converters for cbor2."""
-from datetime import datetime, timezone, date
-from typing import Any, Type, TypeVar
+from datetime import date, datetime, timezone
+from typing import Any, Type, TypeVar, Union
 
 from cbor2 import dumps, loads
 
 from cattrs._compat import AbstractSet
 
 from ..converters import BaseConverter, Converter
+from ..strategies import configure_union_passthrough
 
 T = TypeVar("T")
 
@@ -32,6 +33,7 @@ def configure_converter(converter: BaseConverter):
     )
     converter.register_unstructure_hook(date, lambda v: v.isoformat())
     converter.register_structure_hook(date, lambda v, _: date.fromisoformat(v))
+    configure_union_passthrough(Union[str, bool, int, float, None, bytes], converter)
 
 
 def make_converter(*args: Any, **kwargs: Any) -> Cbor2Converter:

src/cattrs/preconf/cbor2.py

@@ -1,12 +1,13 @@
 """Preconfigured converters for the stdlib json."""
 from base64 import b85decode, b85encode
-from datetime import datetime, date
+from datetime import date, datetime
 from json import dumps, loads
 from typing import Any, Type, TypeVar, Union
 
 from cattrs._compat import AbstractSet, Counter
 
 from ..converters import BaseConverter, Converter
+from ..strategies import configure_union_passthrough
 
 T = TypeVar("T")
 
@@ -36,6 +37,7 @@ def configure_converter(converter: BaseConverter):
     converter.register_structure_hook(datetime, lambda v, _: datetime.fromisoformat(v))
     converter.register_unstructure_hook(date, lambda v: v.isoformat())
     converter.register_structure_hook(date, lambda v, _: date.fromisoformat(v))
+    configure_union_passthrough(Union[str, bool, int, float, None, bytes], converter)
 
 
 def make_converter(*args: Any, **kwargs: Any) -> JsonConverter:

src/cattrs/preconf/json.py

@@ -1,12 +1,13 @@
 """Preconfigured converters for msgpack."""
-from datetime import datetime, timezone, date, time
-from typing import Any, Type, TypeVar
+from datetime import date, datetime, time, timezone
+from typing import Any, Type, TypeVar, Union
 
 from msgpack import dumps, loads
 
 from cattrs._compat import AbstractSet
 
 from ..converters import BaseConverter, Converter
+from ..strategies import configure_union_passthrough
 
 T = TypeVar("T")
 
@@ -36,6 +37,7 @@ def configure_converter(converter: BaseConverter):
     converter.register_structure_hook(
         date, lambda v, _: datetime.fromtimestamp(v, timezone.utc).date()
     )
+    configure_union_passthrough(Union[str, bool, int, float, None, bytes], converter)
 
 
 def make_converter(*args: Any, **kwargs: Any) -> MsgpackConverter:

src/cattrs/preconf/msgpack.py

@@ -9,6 +9,7 @@
 from cattrs._compat import AbstractSet, is_mapping
 
 from ..converters import BaseConverter, Converter
+from ..strategies import configure_union_passthrough
 
 T = TypeVar("T")
 
@@ -66,6 +67,7 @@ def key_handler(v):
     converter._unstructure_func.register_func_list(
         [(is_mapping, gen_unstructure_mapping, True)]
     )
+    configure_union_passthrough(Union[str, bool, int, float, None], converter)
 
 
 def make_converter(*args: Any, **kwargs: Any) -> OrjsonConverter: