MoonBit Regex Matching and Scanning Specification

This repository specifies the MoonBit language behavior of:

• regex match expressions using =~;
• lexmatch expressions over String and StringView, and, with bytes regexes reb"...", over Bytes and BytesView; and
• lexscan expressions over @lexbuf.Lexbuf, @lexbuf.AsyncLexbuf, and @lexbuf.StringScanner.

The specification targets moon 0.1.20261102 and moonc v0.10.19+bee578715, the MoonBit pre-release toolchain of 2026-11-02, which CI installs with install/unix.sh pre-release. The stable latest build of v0.10.19 (+97801ee1b) predates bytes regexes.

The published MoonBit v0.10.7 documentation is linked for comparison. Version differences and publicly reproducible implementation divergences are recorded separately without exposing non-public compiler materials.

Documents

• Scope and conformance
• Regex patterns and matching model
• Regex match expression (=~)
• lexmatch expression
• lexscan expression
• Diagnostics
• Conformance coverage matrix
• Review questions
• Known implementation issues and limitations
• Performance guidance (informative)
• Bytes regex patterns (reb"...")

Writing fast matches

For code generators and agents, in short (details in Performance guidance):

• Test for a fixed string with =~ and an anchored literal: s =~ re"^#", s =~ (re"\.mbt$", before=stem). It compiles to one ==, prefix, suffix or find test, with no automaton.
• Small anchored sets of literals, such as s =~ re"^(if|else|while)$", and anchored literals ignoring ASCII case, such as re"^(?i:host):", are compared as a trie, with no automaton.
• A lexmatch whose cases are all such literals anchored with ^ only, with a catch-all and no as binding, compares all cases in one trie. A single regex case followed by a catch-all compiles like =~.
• Anchor with ^ and $ when a prefix, suffix or whole input is meant; an unanchored pattern is a search.
• Use before and after instead of an as binding when they give what is needed: any as binding makes the match use an automaton.
• Match Bytes and BytesView with reb"..." instead of decoding them; the same rules apply.

Project status

The specification and executable conformance suite are both present. Runtime, typing, diagnostic, warning, formatting, multi-backend, Unicode, streaming, and coverage-audit tests correspond directly to the IDs in the conformance matrix.

The current matrix contains 269 independently testable requirements. The ordinary runtime/type layer contains 146 tests; diagnostic fixtures and format fixtures cover the remaining compile-time requirements and interactions.

Run the complete suite with:

bash scripts/test_all.sh

The full command performs:

• formatting and warning-free project checks;
• debug and release tests across all supported MoonBit backends;
• compile-fail and warning fixtures with diagnostic-code assertions;
• a canonical formatter fixture; and
• a coverage-ID audit that fails if any matrix row lacks a test.

With the baseline toolchain, --target all runs wasm, wasm-gc, js, and native. The separately listed llvm target is experimental and is not runnable unless the toolchain installation includes its LLVM core bundle.

Individual layers can also be run with moon test, bash diagnostics/run.sh, bash scripts/check_format_fixture.sh, and bash scripts/check_coverage_ids.sh.

GitHub Actions runs explicit table, block, and runtime regex backends on wasm, wasm-gc, js, and native, in debug and release mode. Each backend has a separate build directory. Formatting, warning-free type checks, coverage IDs, the formatter fixture, and diagnostic fixtures are also checked.

Temporary expected failures are versioned in FIXME.json. Every listed test is still compiled and run in each applicable backend, target, and mode. A matching failure is XFAIL and does not block CI. A successful test is XPASS and fails CI, prompting removal or narrowing of its FIXME entry. Unlisted failures and failures with a different signature remain blocking. Unknown or renamed test identities and malformed list entries also fail CI.

The CI runner isolates each listed test in a temporary project, preserving helper declarations, so a compiler error in one FIXME cannot prevent ordinary tests from executing. Type checks cover the ordinary project; all isolated FIXME cases are checked by the debug/release test jobs. The tracked sources and test expectations are unchanged by the runner. Local strict commands above continue to run the entire suite without expected-failure allowances.

To reproduce the CI policy locally:

node scripts/ci.mjs check
node scripts/ci.mjs test table js debug
node scripts/ci.mjs test runtime wasm-gc release
bash diagnostics/run.sh --fixme

The diagnostic runner is intentionally strict. A baseline compiler that still has a divergence listed in Known implementation issues and limitations will fail the corresponding fixture until that compiler fix is available in the invoked toolchain.

To test a locally built compiler while keeping the installed moon driver, set SPEC_MOONC_OVERRIDE:

SPEC_MOONC_OVERRIDE=/path/to/moonc bash scripts/test_all.sh