Skip to content

feat(std): match regular expressions with a Pike VM - #439

Merged
0xErwin1 merged 1 commit into
mainfrom
feat/regex-2-vm-pr
Oct 28, 2026
Merged

0xErwin1 merged 1 commit into
mainfrom
feat/regex-2-vm-pr

Conversation

@0xErwin1

Copy link
Copy Markdown
Collaborator

Second slice of std::regex (the parser landed in #438): compiling a pattern and matching it with a Pike VM.

What changes

  • Regex::new(pattern: str): Result<Regex, RegexError> parses and compiles a pattern, so import Regex from "std::regex" works.
  • Compiler: turns the AST into a program of RANGES, ANY_EXCEPT_NEWLINE, SPLIT (priority first; reversed for lazy repetition), JUMP, SAVE, ASSERT and MATCH. Counted repetition expands into copies, and a program past 100 000 instructions is rejected with pattern too large while it is being built.
  • Pike VM:
    • Every thread runs in lockstep over the UTF-8 text, one scalar per step.
    • Each step has a sparse set of visited instructions, which also stops empty loops from running forever.
    • Thread order gives leftmost-first semantics, and a new start thread is added at each position with the lowest priority.
    • Threads are followed with an explicit stack, so deep programs don't use the C call stack.
    • Time is linear in the text for a given pattern, and there is no backtracking.
  • API:
    • isMatch, find and findAll. findAll returns non-overlapping matches and moves forward one scalar after an empty match.
    • Match keeps only byte offsets (start, end, length, isEmpty) and copies the matched text with textIn(haystack).
    • The capture slots for every group are already saved, ready for the next slice.

Evidence

  • ignis test-std regex 32/32. The tests cover literals, classes, . against \n, anchors, \b, alternation priority (a|ab on "ab" gives 0..1), greedy against lazy, counts, UTF-8 (. over a 4-byte scalar), (?i), empty matches in findAll (a* on "baa" gives 0..0, 1..3, 3..3), no match, and pattern too large.
  • Linear time: (a*)*b and (a|aa)*c on 10 000 a finish in 0.1 s.
  • A bug the \b test caught is fixed: the search stopped when a position had no live threads, instead of trying the next start position.
  • Full test-std 638/638 on the stacked branch; check-std and fmt --check are clean.

Regex::new parses and compiles a pattern to a program (RANGES, ANY,
SPLIT with priority, JUMP, SAVE, ASSERT, MATCH; at most 100000
instructions, else 'pattern too large'), and a Pike VM runs every thread in
lockstep over the UTF-8 text, so matching takes time linear in the text for
a given pattern and never backtracks. isMatch, find and findAll give
leftmost-first, non-overlapping matches as byte offsets (Match keeps offsets
only and copies with textIn). Capture slots for every group are already
saved, for the captures slice.
@0xErwin1
0xErwin1 merged commit 6106771 into main Oct 28, 2026
9 checks passed
@0xErwin1
0xErwin1 deleted the feat/regex-2-vm-pr branch October 28, 2026 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant