Repository navigation
feat(std): match regular expressions with a Pike VM - #439
Merged
Merged
Conversation
Regex::new parses and compiles a pattern to a program (RANGES, ANY, SPLIT with priority, JUMP, SAVE, ASSERT, MATCH; at most 100000 instructions, else 'pattern too large'), and a Pike VM runs every thread in lockstep over the UTF-8 text, so matching takes time linear in the text for a given pattern and never backtracks. isMatch, find and findAll give leftmost-first, non-overlapping matches as byte offsets (Match keeps offsets only and copies with textIn). Capture slots for every group are already saved, for the captures slice.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second slice of
std::regex(the parser landed in #438): compiling a pattern and matching it with a Pike VM.What changes
Regex::new(pattern: str): Result<Regex, RegexError>parses and compiles a pattern, soimport Regex from "std::regex"works.RANGES,ANY_EXCEPT_NEWLINE,SPLIT(priority first; reversed for lazy repetition),JUMP,SAVE,ASSERTandMATCH. Counted repetition expands into copies, and a program past 100 000 instructions is rejected withpattern too largewhile it is being built.isMatch,findandfindAll.findAllreturns non-overlapping matches and moves forward one scalar after an empty match.Matchkeeps only byte offsets (start,end,length,isEmpty) and copies the matched text withtextIn(haystack).Evidence
ignis test-std regex32/32. The tests cover literals, classes,.against\n, anchors,\b, alternation priority (a|abon"ab"gives0..1), greedy against lazy, counts, UTF-8 (.over a 4-byte scalar),(?i), empty matches infindAll(a*on"baa"gives0..0, 1..3, 3..3), no match, andpattern too large.(a*)*band(a|aa)*con 10 000afinish in 0.1 s.\btest caught is fixed: the search stopped when a position had no live threads, instead of trying the next start position.test-std638/638 on the stacked branch;check-stdandfmt --checkare clean.