Skip to content

Document errors that come from implementation limits - #310

Open
faresrafat3 wants to merge 1 commit into
hukkin:masterfrom
faresrafat3:tomli-309-impl-limit-errors
Open

faresrafat3 wants to merge 1 commit into
hukkin:masterfrom
faresrafat3:tomli-309-impl-limit-errors

Conversation

@faresrafat3

@faresrafat3 faresrafat3 commented Sep 30, 2026 •

Copy link
Copy Markdown

Fixes #309.

This is the documentation update that #309 ends up calling for. The maintainer of Tomli
agreed in that issue that invalid TOML should stay distinguishable from an error caused
by an implementation limit, and that the RecursionError case needed documenting too:

I think it's good that invalid TOML can be distinguished from an error resulting from
implementation limitations of Tomli. Perhaps this and the RecursionError (caused by
deeply nested inline table or array) need documenting though?

@encukou suggested the wording, and this implements it:

I think it would be enough to say something like "Note that Tomli may raise a different
exception if an implementation-defined limit is exceeded, such as RecursionError for
deeply nested tables or ValueError for extremely large integers."

The note is added to the "Handle invalid TOML" section of the README, right after the
example: TOMLDecodeError covers invalid TOML only, while errors caused by Tomli's own
implementation limits propagate unchanged, because the document may be valid TOML that
this implementation cannot represent.

One wording adjustment against the suggestion above: the note says "deeply nested inline
tables or arrays" rather than "deeply nested tables". The maintainer's comment above
spoke of a "deeply nested inline table or array", so the note names both.

Verification

Checked against master (2.5.0) by running the library from this branch under
CPython 3.12:

>>> import tomli
>>> tomli.loads("a = " + "1" * 4301)
ValueError: Exceeds the limit (4300 digits) for integer string conversion: value has 4301 digits; use sys.set_int_max_str_digits() to increase the limit
>>> tomli.loads("a = " + "[" * 402 + "]" * 402)
RecursionError: TOML inline arrays/tables are nested more than the allowed 400 levels
>>> tomli.loads("a = " + "{x=" * 402 + "1" + "}" * 402)
RecursionError: maximum recursion depth exceeded
>>> tomli.loads("]] this is invalid TOML [[")
tomli._parser.TOMLDecodeError: Invalid statement (at line 1, column 1)

The two errors named in the note are distinct: an oversized integer raises a plain
ValueError, and deep nesting raises RecursionError. Both are limits of the
implementation rather than of TOML, which is why the note calls them
"implementation-defined".

mdformat --check README.md passes locally with the versions pinned in
.pre-commit-config.yaml, and the full CI matrix is green.

Copilot AI balanced review requested due to automatic review settings September 30, 2026 22:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@codecov

codecov Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (43a86ad) to head (47968f4).

Additional details and impacted files
@@            Coverage Diff            @@
##            master      #310   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            4         4           
  Lines          535       535           
  Branches       100       100           
=========================================
  Hits           535       535           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@faresrafat3

Copy link
Copy Markdown
Author

Hi @hukkin — checking in on this PR. It documents the implementation-limit errors that tomli can raise. CI is green and all tests pass. Let me know if there's anything you'd like me to adjust.

@faresrafat3

Copy link
Copy Markdown
Author

Thanks for the review @hukkin! I've updated the PR to document the implementation-limit errors that tomli can raise. CI is green and all tests pass. Let me know if there's anything else you'd like me to adjust.

@encukou

encukou commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator

I think it would be enough to say something like “Note that Tomli may raise a different exception if an implementation-defined limit is exceeded, such as RecursionError for deeply nested tables or ValueError for extremely large integers.”

@faresrafat3

Copy link
Copy Markdown
Author

Done — replaced the expanded explanation with exactly that wording. The worked examples and the ValueError subclassing note are gone; it is now the single note you suggested. Thanks for the steer.

@faresrafat3

Copy link
Copy Markdown
Author

Hi @encukou — the PR now carries exactly the wording you suggested, replacing the expanded explanation:

Note that Tomli may raise a different exception if an implementation-defined limit is exceeded, such as RecursionError for deeply nested tables or ValueError for extremely large integers.

The worked examples and the ValueError subclassing note are gone. It is the single note, nothing else. CI is green and all tests pass.

Ready for review whenever you get to it — happy to adjust further if it still needs trimming.

@faresrafat3
faresrafat3 force-pushed the tomli-309-impl-limit-errors branch from 45cc176 to e83796a Compare October 21, 2026 18:22
TOMLDecodeError is raised only for invalid TOML. Errors that come from
Tomli's own implementation limits propagate unchanged, because the document
may be valid TOML that this implementation cannot represent:

- ValueError for a decimal integer longer than Python's integer string
  conversion limit (4300 digits by default)
- RecursionError for a deeply nested inline table or inline array

Fixes hukkin#309.
@faresrafat3
faresrafat3 force-pushed the tomli-309-impl-limit-errors branch from e83796a to 47968f4 Compare October 21, 2026 18:42
@faresrafat3

Copy link
Copy Markdown
Author

Rebased this onto current master and squashed it into a single commit, so the diff is
now just the one note.

The wording is the one @encukou suggested, with one adjustment: it says "deeply nested
inline tables or arrays" rather than "deeply nested tables", to match the phrase used in
#309.

The commit is now signed with my verified GitHub address, so it is attributed to my
account rather than showing up as an unattributed commit.

The verification section in the description lists the actual output on master, which
includes a limit I had stated wrong before: nested inline arrays raise RecursionError
at 400 levels (MAX_INLINE_NESTING), while nested inline tables hit Python's own
recursion limit. The note itself does not state a number, since both limits are
implementation details that can change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

loads raises ValueError, not TOMLDecodeError, on an invalid document containing a >4300-digit integer

3 participants