Skip to content

Fix unfinished-escape error positions while preserving EOF diagnostics - #326

Open
Levan-Demetrashvili wants to merge 1 commit into
hukkin:masterfrom
Levan-Demetrashvili:fix/312-unfinished-escape
Open

Levan-Demetrashvili wants to merge 1 commit into
hukkin:masterfrom
Levan-Demetrashvili:fix/312-unfinished-escape

Conversation

@Levan-Demetrashvili

Copy link
Copy Markdown

Fixes #312. Related: python/cpython#158566.

An unfinished escape at the end of a basic string or quoted key reports a position one character past EOF. For a document consisting of a quote followed by a backslash, TOMLDecodeError.pos is 3 even though the document has only two characters.

Advance by the length of the escape slice actually consumed. The incomplete escape now reports EOF (pos == len(doc)) and the correct column, while retaining the existing error message. Complete escapes still advance by two characters.

This is an alternative to #314, which reports the backslash's position. Reporting EOF follows the convention already used for unterminated strings and preserves the existing at end of document message.

The regression test adds 22 subcases covering quoted and dotted keys, table and array-of-table names, values inside arrays and inline tables, multiline strings, CRLF normalization, Unicode, a final unpaired backslash after an escaped backslash, and both loads() and binary load(). It checks the normalized document, position, line, column, and error message. All 22 subcases fail on the unchanged source and pass with the fix.

Validation on Linux:

  • Full Tomli unittest suite on CPython 3.12.14, 3.13.5, and a locally built 3.16.0a1+dev debug interpreter; 19 methods on each. The existing Python 3.15+ lazy-import test is skipped on 3.12/3.13.
  • Full suite against a mypyc-compiled CPython 3.12 wheel; 19 methods, with the existing pure-Python lazy-import test skipped.
  • Coverage: 100% statements and branches across the Tomli source.
  • 24,051 differential inputs against the unchanged source, including 733 UTF-8 corpus files and 5,000 Hypothesis examples. Only the expected EOF positions and columns changed; accepted values and rendered error messages were unchanged.
  • Black formatting, isort, flake8, mypy, and git diff --check passed for the relevant source/tests. The complete upstream OS/Python CI matrix has not been run locally.

@codecov

codecov Bot commented Oct 31, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (43a86ad) to head (b6d5c78).

Additional details and impacted files
@@            Coverage Diff            @@
##            master      #326   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            4         4           
  Lines          535       535           
  Branches       100       100           
=========================================
  Hits           535       535           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TOMLDecodeError reports an error position beyond EOF for an unfinished escape

1 participant