Skip to content

fix(core): keep non-UTF-8 bytes of plain G-code byte-for-byte - #15

Merged
jmlweb merged 3 commits into
mainfrom
fix/non-utf8-bytes
Nov 12, 2026
Merged

jmlweb merged 3 commits into
mainfrom
fix/non-utf8-bytes

Conversation

@jmlweb

@jmlweb jmlweb commented Nov 12, 2026

Copy link
Copy Markdown
Owner

Plain uploads were decoded as UTF-8 with U+FFFD replacement and re-encoded as UTF-8, so a kept line with a Latin-1/Windows-1252 byte changed in the output, breaking ADR-0002 (RPT-006).

  • The UTF-8 decoder now reports whether it replaced a byte; if so the file is read as Latin-1 (one code point per byte) and written back as Latin-1. Valid UTF-8 (BOM, literal U+FFFD included) keeps the UTF-8 path. Lossless, so no ADR.
  • New raw-byte property test (random bytes in comment lines) and a raw-byte check over every plain fixture. A mutation forcing UTF-8 fails them.
  • Known limit: report text from non-Latin-1 legacy encodings shows mojibake; bytes are preserved and tower names are ASCII.

Closes TSK-013.

@jmlweb
jmlweb force-pushed the fix/non-utf8-bytes branch from 5cc8de8 to 26dce2e Compare November 12, 2026 18:32
@jmlweb
jmlweb merged commit 22fe99f into main Nov 12, 2026
1 check passed
@jmlweb
jmlweb deleted the fix/non-utf8-bytes branch November 12, 2026 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant