Skip to content

feat: inspect and decode underscore-hex and percent escaped text - #13

Merged
NaNIsaac merged 3 commits into
mainfrom
feat/decode
Oct 29, 2026
Merged

NaNIsaac merged 3 commits into
mainfrom
feat/decode

Conversation

@NaNIsaac

@NaNIsaac NaNIsaac commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

Adds two subcommands for Korean text that arrives as escaped UTF-8 bytes in file names and URLs:

hanfix inspect _ED_8A_B8_EB_A6_AC                     report: type, status, reason, candidate
hanfix decode --type underscore-hex _ED_8A_B8_EB_A6_AC   트리
hanfix decode --type percent %ED%8A%B8%EB%A6%AC          트리
printf '%s' '_ED_8A_B8' | hanfix decode --type underscore-hex

CLI compatibility

inspect and decode are subcommands only as the first argument. Anything else is unchanged:
hanfix -- inspect still converts the word, and the shell functions call hanfix --to en -- $last, so hf
and its Korean twin never reach the new code. tests/cli.rs covers both, plus hanfix ㅑㅜ넫ㅊㅅ → inspect.
The only way an existing call changes is a bare hanfix inspect / hanfix decode meant as English keys to
convert into Hangul; hanfix -- inspect does that now.

Contract (tested)

  • underscore-hex: _ + two upper-case hex digits, bytes 0x80-0xFF only. backup_2026_10_29, hello_bed,
    hello_world and the first _ of report__ED_8A_B8.pdf are kept; __ is never a space.
  • underscore-hex runs also end at a byte that cannot start a UTF-8 character (80-BF, C0, C1, F5-FF); that _XX
    stays text and scanning goes on: IMG_8123.JPG, DSC_9001.JPG, test_99.txt unchanged with exit 0,
    _ED_8A_B8_99.txt -> 트_99.txt, IMG_8123__ED_8A_B8.jpg -> IMG_8123_트.jpg. Inside a character every
    escape is taken, so _ED_8A (incomplete), _ED_C3_A9 and _DB_backup (_DB is a lead byte) still fail.
    Percent is unchanged: every %XX is an escape.
  • Error messages show at most the first 40 characters of the failing run, plus its length.
  • --type given twice is a usage error.
  • percent: %XX, any byte, either case. % without two hex digits is text (100% done, %GG). + is kept.
  • Consecutive escapes are one byte sequence and must be complete, valid UTF-8. Otherwise decode exits 1,
    writes nothing to stdout and one hanfix: error: ... line to stderr. No U+FFFD substitution. The whole
    text is checked before anything is printed.
  • Text with no escapes: decode prints it unchanged, exit 0.
  • inspect exits 0 on any text (plain, invalid, mixed); valid / ambiguous (both kinds present) /
    invalid, with a rule-based reason, no confidence numbers.
  • Input on stdin is read strictly as UTF-8 for both subcommands (exit 1 otherwise); the existing conversion
    keeps its lossy read.
  • Usage errors (missing or unknown --type, unknown option) exit 2, as before.

Files

  • src/encoding.rs: detection, validation and decoding as pure functions with unit tests; no new crates.

  • src/main.rs: subcommand dispatch at the top of run(), help and usage text, read_stdin() shared with
    the existing path. usage_error uses eprintln! (same output) so cargo clippy -- -D warnings passes.

  • tests/cli.rs: runs the built binary: stdout/stderr/exit contract, stdin, usage errors, subcommand words
    as text, existing conversions.

  • README.md, README.ko.md: new "Decode escaped Korean text" section, usage, --type, exit status, and a
    PowerShell Rename-Decoded function that previews old -> new by default and renames only with
    -Apply, skipping names that fail to decode, do not change, are not valid file names, or would overwrite.

  • tests/shells.ps1: extracts that README block, checks the Korean README has the same code, and runs it
    on a temp folder (preview changes nothing; -Apply renames only the decodable, free name).

  • Rename-Decoded first probes hanfix decode --type percent -- %41 and throws hanfix too old: update with hanfix --update unless it prints A: hanfix 0.2.7 and older read decode ... as text and exit 0, so without the probe the example would rename every file. tests/shells.ps1 checks this with a stand-in for the old binary (a function calling hanfix -- @args); nothing is renamed.

  • inspect adds Note: lower-case _xx is not treated as an escape for runs such as _ed_8a_b8.

  • Rename-Decoded also skips results that are more than one line or contain control characters (no CR to LF
    rewrite), end in a dot or space (Windows would drop them), or are a Windows device name; it collects every
    name in the folder and every name planned in the run, ignoring case, so the preview lists exactly what
    -Apply does. Tested in tests/shells.ps1 (percent folder: preview equals apply), also run in Windows
    PowerShell 5.1.

  • Version 0.2.8.

Not in this PR: mojibake (planned next) and the release.

local-ci: 7504d53 | cargo test --release --locked (10 unit + 7 CLI pass), cargo clippy --all-targets --locked -- -D warnings clean, pwsh tests/shells.ps1 with this build first on PATH (3 ok), README example in Windows PowerShell 5.1 (preview correct) on Windows | not run: tests/shells.sh (bash/zsh/fish, Linux only; init scripts untouched), macOS, Termux

@NaNIsaac
NaNIsaac merged commit 96b6866 into main Oct 29, 2026
21 checks passed
@NaNIsaac
NaNIsaac deleted the feat/decode branch October 29, 2026 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant