Skip to content

Fix semicolon-free imports and MDX parsing - #3

Merged
dav-is merged 1 commit into
mainfrom
fix/parse-imports-and-mdx
Sep 25, 2026
Merged

dav-is merged 1 commit into
mainfrom
fix/parse-imports-and-mdx

Conversation

@dav-is

@dav-is dav-is commented Sep 25, 2026

Copy link
Copy Markdown
Member

Summary

Fixes how parseImportsAndComments reads imports, comments and MDX structure, and makes it linear. Found while building @fairgarden/design's demos, most of which omit semicolons.

  • One statement scanner, following the import grammar. Semicolon-free imports, imports split across lines, and export … from are read correctly. Stray words and strings no longer become fake imports.
  • MDX follows its real parsing rules: code fences, ESM blocks and prose are tracked as remark-mdx does.
  • Performance: parsing now scales linearly and is about 8.3× faster than main on the real corpus. The branch had introduced quadratic cases, and main already had some.

Statements

findModuleStatementEnd replaces two near-duplicate loops and skips strings, template literals and comments whole.

  • Before the module path: a line break never ends a statement, so import React,\n { useState } …, import\n * as React … and export type\n { A } from … work. Only binding names, type / as, *, commas and one { … } may appear, and from counts only when a quoted path follows it.
  • Anything else before the path is ordinary code. That includes a reserved word, =, (, ; or the end of the file. So an incomplete import can't swallow the rest of the file, and import x = require() is left alone.
  • After the path: ;, end of line or a trailing comment ends the statement. A with / assert { … } clause may continue on the next line.
  • isExportFromStatement accepts only export [type] * [as x] from '…' and export [type] { … } from '…'.

Fixed along the way (several were on main):

  • Fake imports:
    • From from in an exported body, JSX text, strings or comments. For example, <p>Rows loaded from './rows.json'</p> broke the build, and // Adapted from "React docs" became an uninstallable dependency in the sandbox package.json.
    • From escaped quotes: the escape check compared against a two-character string that never matched.
    • From import.meta, which was read as an import.
    • From an apostrophe in a comment inside import braces, which swallowed later imports.
  • Garbled output from two stripped comments on one line: the rollback removed the source line instead of the output line.
  • A dropped last line when a file ends in a stripped comment with no trailing newline. This affected MDX fences with filename= or variant=.
  • Comment positions and endings: comments glued to the closing quote, or between the path and ;, are stripped, and CRLF endings are kept.

MDX

  • Code fences follow CommonMark as MDX configures it:
    • a fence opens with a run of at least 3 ` or ~ at the start of a line
    • a backtick info string can't contain a backtick
    • it closes only with the same marker, at least as long as the opener, on a line of its own; an unclosed fence runs to the end of the file
  • Inline code is not a fence. Other backtick runs, including three or more mid-line, are inline code. Before, they flipped fence tracking.
  • Fences may be indented any amount, because MDX disables indented code, and micromark only caps fence indentation when indented code is on. .md files don't take the Markdown path at all.
  • ESM blocks follow mdxjs-esm.
    • Start: isMdxEsmStart is the only check, and it requires import or export followed by a space at the start of a line. So import{Fake}from './fake' in prose stays text.
    • End: a block ends at a blank line only where its code is complete (bracket depth, strings, templates and comments are tracked). Code that never completes, which MDX rejects, ends at its first blank line, so the scan stays linear.
  • Dynamic import() is detected anywhere inside an ESM block, but not in prose, fences or JSX expressions. remark-mdx treats expressions as render-time code.
  • Parity: detected imports match remark-parse + remark-mdx's ESM on all 189 MDX pages across fg-docs and @fairgarden/design, plus the fixtures.
  • No micromark dependency: the rules are implemented directly because this module also runs in the browser (via absolutizeImports).

Performance

The output is unchanged by the rework: byte-identical across 2,532 inputs × 3 option sets, including CRLF copies.

main this PR
Corpus (10.1 MB, 3 option sets) 3,990 ms 478 ms
Large TSX, 1× → 16× 22 ms → 5.3 s 1.5 ms → 49 ms
Minified single line, 1× → 16× 5 ms → 97 ms 1.3 ms → 49 ms
Unclosed export { braces 22 ms 1.7 ms

optimization.test.ts replaces the 500 ms wall-clock asserts with complexity guards. Each parses an input at n and 16n and fails if the time grows 64× or more; linear growth is about 16×. They fail against the pre-rework parser.

Behavior check against main

  • fg-docs (1,366 files): every difference is a fix: fake imports removed, and imports main missed are found.
  • @fairgarden/design demos (829 files): all 617 semicolon-style files are identical. The semicolon-free files change as intended (for example, relative imports detected go from 1 to 209).

Test changes to review

  • Snapshots updated: three inline snapshots in enhanceCodeEmphasis.test.ts recorded the dropped-last-line and unstripped-comment bugs. The "wrap complete elements" assertion only passed because an unstripped comment contained <mark>.
  • Export-from test updated: "export-from with newline instead of semicolon" had pinned the old output (second export dropped).
  • Inline-code fixture fixed: a closing ``` alone on its own line opens a fence in CommonMark, so the fixture now uses a real code span.
  • Test removed: the one asserting that /* @highlight */ before an import's ; stays, contrary to removeCommentsWithPrefix's docs.
  • AGENTS.md: the stale warnings about trailing // @highlight and createDemo(import.meta.url, …, { are removed, since both bugs are fixed.

Known limitations

  • An import / export line that directly continues a prose paragraph is read as ESM; MDX reads it as text. Indented import … lines are also read as ESM (deliberate, per the docstring). No real page does either.
  • JSX inside an ESM export that spans a blank line ends the block early.
  • import() inside a template ${…} isn't detected, in .ts files too.
  • Garbage collection: above about 30 MB of allocations per parse, GC scavenges add a step in timing, and growth is linear again after it.

Verification

Check Result
pnpm test --run 4,950 passing (+77)
pnpm typescript / pnpm eslint / prettier clean
pnpm docs:lib / pnpm docs:build ✅
pnpm docs:validate 0 files updated

- Scan statements so imports without semicolons end at a line break,
  a trailing comment or an incomplete statement instead of swallowing
  the code after them
- Track MDX code fences (backtick and tilde, run length, indentation)
  and ESM block boundaries, so prose, inline code, `import()` and
  `import.meta` are no longer read as imports
- Keep the last line when a stripped comment ends a file without a
  trailing newline, and keep emphasis comments after semicolon-free
  imports
- Preserve CRLF line endings and escaped quotes when stripping comments
- Parse in a single pass so parse time grows linearly, guarded by new
  optimization tests
- Update the enhanceCodeEmphasis snapshots and drop the AGENTS.md
  warnings these fixes make stale

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@dav-is dav-is added the bug Something isn't working label Sep 25, 2026
@vercel

vercel Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
fairgarden-docs Ready Ready Preview Sep 25, 2026 6:35pm UTC

@dav-is
dav-is marked this pull request as ready for review September 25, 2026 18:34
@dav-is
dav-is merged commit 2c5048f into main Sep 25, 2026
14 of 15 checks passed
@dav-is
dav-is deleted the fix/parse-imports-and-mdx branch September 25, 2026 21:29

This branch was successfully deployed

1 active deployment
Preview — b501488f Deployed Sep 25, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant