Skip to content

fix: store a list of models that admits a None - #1977

Open
shcheklein wants to merge 1 commit into
fix/nullable-array-item-spellingsfrom
fix/optional-model-array-items
Open

fix: store a list of models that admits a None#1977
shcheklein wants to merge 1 commit into
fix/nullable-array-item-spellingsfrom
fix/optional-model-array-items

Conversation

@shcheklein

Copy link
Copy Markdown
Contributor

Stacked on #1976.

is_chain_type accepts list[Child | None], and list[Child] writes, but
building the schema raised:

class Child(dc.DataModel):
    x: int

class Holder(dc.DataModel):
    vals: list[Child | None]

chain.map(h=..., output=Holder)
# TypeError: Cannot recognize type <class 'Child'>

The check that maps a model element to JSON looked at the annotation as written,
and the union arm was all that hid the model. It asks what the arm holds now.

With #1975's flatten fix already in place, all three orders round-trip:

[Child(x=1), None]  ->  [Child(x=1), None]
[None, Child(x=2)]  ->  [None, Child(x=2)]
[None, None]        ->  [None, None]

What stays refused, deliberately

Only a plain Model | None is admitted. Two shapes are stored wrong rather than
not at all, so they keep raising:

annotation why
list[Annotated[Child, "meta"] | None] rebuilding a nested model does not look through Annotated, so it comes back as [None, {'x': 1}]
list[RootModel[int] | None] a root model dumps to its bare value and the reader rebuilds one only from a mapping, so it comes back as [None, 1]

list[RootModel[int]] already reads back as bare ints on main — that is
pre-existing and left alone; what this must not do is admit a spelling main
refuses and then store it wrong.

Every slot of a fixed tuple is checked too, since they share the one column:
tuple[A | None, Annotated[B, "meta"] | None] is refused on the strength of its
second slot, while tuple[A | None, B | None], whose slots both read back as
models, is allowed. A whole-model read hides this class of bug, because Pydantic
revalidates the dict — only a leaf read shows it.

Compatibility

Nothing that already wrote is affected: every annotation here raised before, so
there is no stored form to stay compatible with. Storage is unchanged across the
fourteen annotations checked in #1975, and the declared type changes only for
list[Model | None], which had none.

Stack

  1. fix: keep None in an array wherever it sits #1975None anywhere in an array
  2. fix: recognize every spelling of a nullable array element #1976 — the spellings of a nullable element type
  3. this PR — a list of models that admits a None

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 2, 2026

Copy link
Copy Markdown

Deploying datachain with  Cloudflare Pages  Cloudflare Pages

Latest commit: b1b6e6e
Status: ✅  Deploy successful!
Preview URL: https://5a52a799.datachain-2g6.pages.dev
Branch Preview URL: https://fix-optional-model-array-ite.datachain-2g6.pages.dev

View logs

@codecov

codecov Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

is_chain_type accepts list[Child | None] and list[Child] writes, but
map(output=...) raised "Cannot recognize type Child" while building the schema:
the union arm hid the model from the check that maps a model element to JSON.
Ask that check what the arm holds as well.

Only a plain Model | None is admitted. An Annotated[Model, ...] | None would be
stored and then handed back as plain dicts, because rebuilding a nested model
does not look through Annotated, and a root model dumps to whatever it wraps
where the reader rebuilds one only from a mapping -- list[RootModel[int]] reads
back as bare ints on main already. Both keep raising rather than storing
something that reads wrong.

Every slot of a fixed tuple is checked, since they share the one column: a slot
that cannot be read back as a model still refuses the annotation.

Nothing that already wrote is affected -- these raised before, so there is no
stored form to stay compatible with.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@shcheklein
shcheklein force-pushed the fix/optional-model-array-items branch from 1972716 to b1b6e6e Compare September 2, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant