Skip to content

fix(util): handle Thai zero ("ศูนย์") in text_to_num floating-point parsing - #1503

Merged
bact merged 6 commits into
PyThaiNLP:mainfrom
Palmkonde:fix/text-to-num-zero-handling
Sep 16, 2026
Merged

bact merged 6 commits into
PyThaiNLP:mainfrom
Palmkonde:fix/text-to-num-zero-handling

Conversation

@Palmkonde

Copy link
Copy Markdown
Contributor

What do these changes do

Fixes text_to_num()/words_to_num()/thaiword_to_num() corrupting, mis-parsing, or crashing on any spelled-out Thai floating-point number that contains a zero digit (ศูนย์).

What was wrong

ศูนย์ (zero) is deliberately excluded from the _digits table in _check_is_thainum() (comment: "excluded as a special case"), but no special-case handling was ever added downstream. Every check in text_to_num()'s loop that gated on _check_is_thainum(word)[0] therefore treated a bare zero as ordinary text, which:

  • desynchronized the loop's adjacency bookkeeping (last_index), silently dropping or misplacing digits after a zero, and
  • could flush an empty digit buffer through words_to_num()/thaiword_to_num(""), raising an unhandled ValueError.

A naive fix (adding ศูนย์ to the digit table outright) would have false-positived on ordinary Thai words that merely contain ศูนย์ as a prefix, e.g. ศูนย์กลาง (center) or ศูนย์รวม (hub), converting them into "0".

How this fixes it

  • _check_is_thainum() now takes optional token-context (tokens, i, thainum) and classifies a bare ศูนย์ token as numeric only when it's directly adjacent to an active decimal run (preceded by จุด in the accumulator, or immediately followed by จุด). Called without context (its original use classifying dictionary words), behavior is unchanged.
  • text_to_num() computes this classification once per loop iteration (isthainum) instead of re-calling _check_is_thainum() up to three times per token.
  • Extracted the buffer-flush logic into _flush(), which only converts the accumulated buffer through words_to_num() when it actually contains a real digit; otherwise it re-emits the buffered words as literal text. This also fixes จุด (decimal point) being misclassified as a numeral unit when it appears with no real digit nearby (e.g. จุดศูนย์กลาง..., "central point"), without needing a separate จุด-specific guard.

Verified against all 9 cases from the issue (5 floating-point-with-zero cases + 4 compound-word false-positive guards), plus the existing docstring examples, tests.core (205 tests) and tests.compact (233 tests total) across Python 3.9/3.10/3.12/3.13 via tox. black, flake8, and ruff check are clean on the changed files.

Fixes #1502

Your checklist for this pull request

  • Passed code styles and structures
  • Passed code linting checks and unit test

@bact bact added the bug bugs in the library label Sep 16, 2026
Comment thread pythainlp/util/wordtonum.py Outdated
bact suggested passing just the next token instead of threading the
full token list and current index through _check_is_thainum; thainum
still needed to detect "ศูนย์" coming after "จุด".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@bact
bact merged commit bab3b43 into PyThaiNLP:main Sep 16, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug bugs in the library

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: pythainlp.util.text_to_num() silently corrupts, mis-parses, or crashes on Thai numerals containing "ศูนย์"

2 participants