Skip to content

lexer: pre-existing PUC 5.3 divergences — bare CR line counting, \ddd escapes > 255 #410

Description

@davydog187

Found during automated review of PR #399 (all confirmed pre-existing on main — the PR does not change them):

  1. Bare \r inside a --[[ ]] comment does not advance the line counter. Lexer.tokenize("--[[a\rb]]\rx") reports x at line 2; PUC's read_long_stringinclinenumber counts it, so x is line 3. scan_long_string has ?\r clauses; the multi-line comment scanner does not.
  2. Bare \r inside a short string is accepted as a literal character. "s='a\rb' x" lexes; PUC's read_string raises "unfinished string" for both \n and \r. The line counter also doesn't advance, so line numbers drift in CR-only-EOL files.
  3. \ddd decimal escapes are cut short instead of rejected. "\256" lexes as \25 followed by literal 6; PUC raises "decimal escape too large". (read_decimal_escape/3 stops before the value can exceed 255, which also made the old value > 255 guard dead code.)

All three are byte-level lexer conformance issues; worth fixing together while the scanners are warm from #399.

🤖 Filed from an automated review pass.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions