[haripetrov-stack]

Description

Markdown import decides whether a *, _, ~~ or == run can open or close formatting by looking at the characters on either side of it. The punctuation and whitespace classes listed only ASCII characters, so an en dash, a curly quote, a currency sign or an emoji next to a run counted as a letter. See –**[here](https://example.com)** now. imported as plain text, and its next export escaped the asterisks.

CommonMark 0.31.2 counts every character in the Unicode P and S categories as punctuation, and Zs plus tab, LF, FF and CR as whitespace. Import and the typing shortcuts now use those sets, reading whole code points so astral characters such as 😀 are classified correctly. The backslash, missing from the old ASCII list, now counts as punctuation too.

Export has to follow: Lexical wrote shapes like foo**“bar”**baz, whose markers no longer flank, and 7 of the MarkdownEscaping.test.ts round trips from #9481 failed. The exporter now writes the letter beside such a marker as a numeric character reference (foo**“bar”**baz), which every CommonMark reader decodes to the same text. It writes none inside code or transformer output. A raw backslash right before a marker, or before a character that export escapes with shouldPreserveNewLines, is escaped too. The references are chosen so that micromark, which @lexical/mdast imports with and which reads an emoji as a letter, formats the result the same way (😀**“b”**).

A bare URL, www. host or email that GFM readers would link needs care: a reference inside it changes the link, and GFM readers take markers and code spans after it into the link. When either would happen, export writes the . of www. or the : after the scheme as a reference, so that no reader links it and the text and formatting come through (**www.x.co**a, see https://x.com/a**“x”**). An email that holds a reference has its @ written as @, so that micromark links the whole decoded address.

Import fixes that the new flanking exposed:

• Emphasis that closes before a later code span is applied before that span, so its markers no longer split apart (**a.**x**`c`**).
• A link or other text match no longer takes part of any code span, not only the first (Bug: Equation $ parsing should not bind tighter than code formatting #8687), and the link search stays bounded on lines with many [ or code spans.
• A delimiter run cut by an earlier match reads as it did in the whole text (*)**a*). Text beside a text match that does not cut a run still reads the match's edge as the edge of a line, as on main, so a custom syntax that ends in a letter (user:bob**(admin)**) keeps the formatting after it.
• An escaped backtick before more backticks no longer stops the rest of the run from opening a code span (x\``bb`x), which export writes for a plain backtick before inline code.

Import now finds a line's code spans, emphasis and links, and the matches of other text-match transformers whose patterns do not look behind, once rather than again after each match, and imports the text after each match in a loop, in the same order as before. Long lines import faster than on main, for example 4 s instead of 9 s for 'a **b** [c](d) `e` _f_ ' repeated 1,600 times, and very long paragraphs no longer overflow the stack.

Markdown such as **你好。**世界, which main read as bold, now imports with literal asterisks, as CommonMark, GFM, micromark and markdown-it show it; a CJK-friendly rule for both importers is left for a follow-up.

Known limitations:

• The rule of 3 still allows pairings by remaining run length, as on main (Bug: Markdown export produces invalid markdown for overlapping formats #4895).
• Some overlapping formats around escaped punctuation on both sides of a marker do not round-trip.
• Emphasis can still pair across a link's label, as in x~~[_](u)[~~](u), as on main.
• A literal == in text is not escaped, so it can pair with a highlight marker that both opens and closes. With Unicode punctuation that now includes a highlight next to an emoji or symbol, not only next to ASCII punctuation as on main.
• Escapes and references that export writes inside a bare URL, such as \_, become part of the link in GFM readers, as on main.
• \v beside a marker cannot be written so that both CommonMark and micromark readers agree.

Reported in sveltia/sveltia-cms#1107.

Closes #9407
Closes #9410
Closes #9443

Changeset

.changeset/markdown-unicode-punctuation-flanking.md, a [fix] patch for @lexical/markdown. Its Verify: paragraph covers stored Markdown such as **“bar”**baz and a**\_x**, which now imports with literal asterisks as CommonMark readers show it, and the new character references in export.

Test plan

Command: pnpm exec vitest run --project unit packages/lexical-markdown

Before

With main merged into the original PR head (5debdb3):

     × preserves existing formatted markers after plain text: bold 15ms
     × preserves existing formatted markers after plain text: italic 2ms
     × preserves existing formatted markers after plain text: strikethrough 3ms
     × preserves formatted backslashes next to plain text: bold 4ms
     × preserves formatted backslashes next to plain text: italic 2ms
     × preserves formatted backslashes next to plain text: strikethrough 2ms
     × preserves formatted backslashes next to plain text: bold+italic 2ms
 Test Files  1 failed | 8 passed (9)
      Tests  7 failed | 844 passed (851)

After

 Test Files  9 passed (9)
      Tests  1055 passed (1055)

The new tests cover Unicode and astral punctuation, spec whitespace, character references read back by both @lexical/markdown and @lexical/mdast, formatting beside bare URLs and emails, code-span and link precedence, cut delimiter runs and custom text-match transformers. Import was also compared with markdown-it on random inputs, where it agrees on many more of them than main does, and export round trips were checked through both importers and cmark-gfm on random paragraphs, with none that main got right and this gets wrong.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
lexical	Ready	Preview	Nov 3, 2026 12:09am UTC
lexical-playground	Ready	Preview	Nov 3, 2026 12:09am UTC

[pkg-pr-new]

lexical

npm i https://pkg.pr.new/lexical@9430

@lexical/a11y

npm i https://pkg.pr.new/@lexical/a11y@9430

@lexical/clipboard

npm i https://pkg.pr.new/@lexical/clipboard@9430

@lexical/code

npm i https://pkg.pr.new/@lexical/code@9430

@lexical/code-core

npm i https://pkg.pr.new/@lexical/code-core@9430

@lexical/code-prism

npm i https://pkg.pr.new/@lexical/code-prism@9430

@lexical/code-shiki

npm i https://pkg.pr.new/@lexical/code-shiki@9430

@lexical/compiler

npm i https://pkg.pr.new/@lexical/compiler@9430

@lexical/devtools-core

npm i https://pkg.pr.new/@lexical/devtools-core@9430

@lexical/dragon

npm i https://pkg.pr.new/@lexical/dragon@9430

@lexical/eslint-plugin

npm i https://pkg.pr.new/@lexical/eslint-plugin@9430

@lexical/extension

npm i https://pkg.pr.new/@lexical/extension@9430

@lexical/fast-check

npm i https://pkg.pr.new/@lexical/fast-check@9430

@lexical/file

npm i https://pkg.pr.new/@lexical/file@9430

@lexical/hashtag

npm i https://pkg.pr.new/@lexical/hashtag@9430

@lexical/headless

npm i https://pkg.pr.new/@lexical/headless@9430

@lexical/history

npm i https://pkg.pr.new/@lexical/history@9430

@lexical/html

npm i https://pkg.pr.new/@lexical/html@9430

@lexical/internal

npm i https://pkg.pr.new/@lexical/internal@9430

@lexical/link

npm i https://pkg.pr.new/@lexical/link@9430

@lexical/list

npm i https://pkg.pr.new/@lexical/list@9430

@lexical/mark

npm i https://pkg.pr.new/@lexical/mark@9430

@lexical/markdown

npm i https://pkg.pr.new/@lexical/markdown@9430

@lexical/mdast

npm i https://pkg.pr.new/@lexical/mdast@9430

@lexical/offset

npm i https://pkg.pr.new/@lexical/offset@9430

@lexical/overflow

npm i https://pkg.pr.new/@lexical/overflow@9430

@lexical/plain-text

npm i https://pkg.pr.new/@lexical/plain-text@9430

@lexical/react

npm i https://pkg.pr.new/@lexical/react@9430

@lexical/rich-text

npm i https://pkg.pr.new/@lexical/rich-text@9430

@lexical/selection

npm i https://pkg.pr.new/@lexical/selection@9430

@lexical/table

npm i https://pkg.pr.new/@lexical/table@9430

@lexical/tailwind

npm i https://pkg.pr.new/@lexical/tailwind@9430

@lexical/text

npm i https://pkg.pr.new/@lexical/text@9430

@lexical/utils

npm i https://pkg.pr.new/@lexical/utils@9430

@lexical/y

npm i https://pkg.pr.new/@lexical/y@9430

@lexical/yjs

npm i https://pkg.pr.new/@lexical/yjs@9430

commit: 7184aaa

[meta-code-review]

Thanks for this. The change looks good to me: classifying punctuation and whitespace with the Unicode sets follows CommonMark 0.31.2, the new tests fail on main and pass here, and the exported references read back through both @lexical/markdown and @lexical/mdast. A few suggestions, all optional and non-blocking:

1. Stored CJK Markdown (a call for maintainers). Markdown that main exported and read back as bold, such as **注意：**请先登录, now imports with literal asterisks, and saving it again writes \*\*注意：\*\*请先登录. That is what CommonMark specifies, and the changeset's Verify: already says so. Bold labels that end in full-width punctuation right before CJK text are common in Chinese and Japanese writing, so it may be worth shipping the CJK-friendly follow-up in the same release. CONTRIBUTING.md files a behaviour-changing fix as [fix] with Verify:, as you did. AGENTS.md (Backwards Compatibility) describes [breaking] with Migration:/Detect: for changes callers depend on, if maintainers prefer that category here.
2. Edge case right after a link (packages/lexical-markdown/src/importTextFormatTransformer.ts, findPairAtEdge, lines 161–195). [x](u)**:*a.** imports on main as x, *, italic :a., reading the run after the link beside the edge of a line as the description intends, but here as x, **:, italic a., *. The search for the pair that the run after the link opens stops before the scan's next emphasis (end), but that pair can contain it. Returning false (so the rest is scanned on its own) when end comes from the next emphasis or code span would keep the edge reading. It is a narrow shape.
3. Export cost (packages/lexical-markdown/src/MarkdownExport.ts, findSpoiledLinks, line 897). Any paragraph whose text contains www., :// or @ builds a masked copy and three typed arrays, even when no reference or marker touches a link. For a 300-paragraph document with a bare URL in each paragraph, $convertToMarkdownString takes about 30 ms against about 7.5 ms on main (12.6 ms without the URLs). An early exit when no link run sits next to a marker or reference would help apps that export on every change.
4. Test plan. The ### Before block shows the original head failing the MarkdownEscaping.test.ts round trips. AGENTS.md (Commits and Pull Requests) asks for the real failing output before the change, so pasting main's run of the new tests would show what they catch.

[meta-code-review]

Optional: when no pair is found in prefixes that end before end, this keeps the whole-line scan, but the pair the edge run opens can contain the next emphasis. [x](u)**:*a.** imports on main as x, *, italic :a. (the run after the link read beside the edge of a line, as the description intends) and here as x, **:, italic a., *. Returning false when end came from the next emphasis or code span, so the rest is scanned on its own, would keep the edge reading.

[meta-code-review]

Optional: any text piece with www., :// or @ sends the whole paragraph through the masked copy and typed arrays below, even when no reference or marker touches a link. A 300-paragraph document with one bare URL per paragraph exports in about 30 ms against about 7.5 ms on main. An early exit when no link run is next to a marker or reference would recover most of that.

[meta-code-review]

Thanks for this! The change looks good to me: classifying punctuation and whitespace with the Unicode sets follows CommonMark 0.31.2, the new tests fail on main and pass here, and the exported references read back through both @lexical/markdown and @lexical/mdast. A few suggestions, all optional and non-blocking:

1. Quadratic escape with shouldPreserveNewLines (packages/lexical-markdown/src/MarkdownExport.ts, line 566). /\\+(?=[*_`~])|[*_`~]/g retries \\+ from every backslash of a run that no marker follows, so exporting a text node of 16,000 backslashes takes about 570 ms, against 0.1 ms on main. Matching the whole run and checking the character after it gives the same output in linear time:

   output = output.replace(
     /\\+|[*_`~]/g,
     (escaped, offset: number, text: string) =>
       escaped[0] !== '\\'
         ? `\\${escaped}`
         : /[*_`~]/.test(text[offset + escaped.length] ?? '')
           ? escaped + escaped
           : escaped,
   );

2. Stored CJK Markdown (a call for maintainers). Markdown that main exported and read back as bold, such as **注意：**请先登录, now imports with literal asterisks, and saving it again writes \*\*注意：\*\*请先登录. That is what CommonMark specifies, and the changeset's Verify: says so. Bold labels that end in full-width punctuation right before CJK text are common in Chinese and Japanese writing, so it may be worth shipping the CJK-friendly follow-up in the same release.

3. Export cost beside bare URLs (MarkdownExport.ts, spoilLinks, line 872). Any paragraph whose text contains www., :// or @ builds the masked copy and the prefix sums even when no reference or marker touches a link; 300 such paragraphs export in about 11 ms against about 2 ms on main. An early exit for that case would help apps that export on every change.

4. A run right after a link (packages/lexical-markdown/src/importTextFormatTransformer.ts, findPairAtEdge, lines 153–197). [x](u)**:*a.** imports on main as x, *, italic :a., reading the run after the link beside the edge of a line as the description says, but here as x, **:, italic a., *. Returning false when end comes from the next emphasis or code span, so that the rest is scanned on its own, would keep the edge reading. It is a narrow shape.

5. Test plan. The ### Before block shows the original head merged with main failing the MarkdownEscaping.test.ts round trips. AGENTS.md (Commits and Pull Requests) asks for the real failing output of the command before the change, so pasting main's run of the new tests would show what they catch.

[meta-code-review]

Optional: \\+(?=[*_~])is retried from every backslash of a run that no marker follows and backtracks through the rest of it each time, so this is quadratic in the run (16,000 backslashes: about 570 ms, against 0.1 ms onmain`). Matching the whole run and checking the next character gives the same output in linear time:

Suggested change

	output = output.replace(/\\+(?=[*_`~])|[*_`~]/g, escaped =>
	escaped[0] === '\\' ? escaped + escaped : `\\${escaped}`,
	);
	output = output.replace(
	/\\+|[*_`~]/g,
	(escaped, offset: number, text: string) =>
	escaped[0] !== '\\'
	? `\\${escaped}`
	: /[*_`~]/.test(text[offset + escaped.length] ?? '')
	? escaped + escaped
	: escaped,
	);

[meta-code-review]

Nit: when no pair is found before end and end comes from the next emphasis or code span, this keeps the whole-line scan, which read the edge run beside the text match. So [x](u)**:*a.** imports as x, **:, italic a., * instead of main's x, *, italic :a.. Returning false here in that case (so the rest is scanned on its own) would keep the edge reading the description promises.

[meta-code-review]

Thanks for this! The change looks good to me: classifying punctuation and whitespace with CommonMark 0.31.2's Unicode sets is what the spec asks for, the new tests fail on main and pass here, and the exported references read back through both @lexical/markdown and @lexical/mdast. A few suggestions, all optional and non-blocking:

1. Quadratic escape with shouldPreserveNewLines (packages/lexical-markdown/src/MarkdownExport.ts, line 566). /\\+(?=[*_`~])|[*_`~]/g retries \\+ from every backslash of a run that no marker follows, so exporting a text node of 16,000 backslashes takes about half a second, against well under a millisecond on main. Matching the whole run and checking the character after it gives the same output in linear time:

   output = output.replace(
     /\\+|[*_`~]/g,
     (escaped, offset: number, text: string) =>
       escaped[0] !== '\\'
         ? `\\${escaped}`
         : /[*_`~]/.test(text[offset + escaped.length] ?? '')
           ? escaped + escaped
           : escaped,
   );

2. Stored CJK Markdown (a call for maintainers). Markdown that main exported and read back as bold, such as **注意：**请先登录, now imports with literal asterisks, and saving it again writes \*\*注意：\*\*请先登录. That is what CommonMark specifies and how @lexical/mdast already reads it, and the changeset's Verify: paragraph says so. Bold labels that end in full-width punctuation right before CJK text are common in Chinese and Japanese writing, so it may be worth shipping the CJK-friendly follow-up in the same release.

3. Test plan. The ### Before block shows the original head merged with main failing the MarkdownEscaping.test.ts round trips. AGENTS.md (Commits and Pull Requests) asks for the real failing output of the command before the change, so pasting main's run of the new tests would show what they catch.

The CI workflows have not run on this head yet; they are waiting for a maintainer to approve them for the fork.

[meta-code-review]

Optional, non-blocking: \\+(?=[*_~])is retried from every backslash of a run that no marker follows, so this is quadratic in the run (16,000 backslashes take about half a second to export, against well under a millisecond onmain`). Matching the whole run and checking the character after it gives the same output in linear time:

output = output.replace(
  /\\+|[*_`~]/g,
  (escaped, offset: number, text: string) =>
    escaped[0] !== '\\'
      ? `\\${escaped}`
      : /[*_`~]/.test(text[offset + escaped.length] ?? '')
        ? escaped + escaped
        : escaped,
);

[meta-code-review]

Thanks for the careful work here. Following CommonMark's Unicode punctuation and whitespace fixes real import bugs (See –**[here](https://example.com)**, **a.**\x, the _ shortcut after – or “), and the new tests cover both importers and the round trip.

A maintainer decision before merge: CJK content that main imports as bold

Stored Markdown such as **注意：**请先登录, - **要点：**说明文字 or 请**“注意”**这里 imports as bold on main, and it is what main's exporter writes for that formatting. With this change it imports with literal **, and the next export escapes them (\*\*注意：\*\*请先登录), so opening and saving such a document turns the bold into visible asterisks. This follows CommonMark 0.31.2 (： is Unicode punctuation, so the closing ** between ： and 请 is not right-flanking), matches GFM, markdown-it and @lexical/mdast, and the changeset's Verify: paragraph names it, so it is not a bug in the implementation. Because AGENTS.md's Backwards Compatibility section asks that existing content keep working, a maintainer should choose one of: merge it as the [fix] patch and leave the CJK-friendly rule to a follow-up; add the CJK-friendly amendment first (as commit 454bf81 did before it was reverted); or keep strict reading and ship it as a [breaking] minor changeset with Migration: and Detect:. (packages/lexical-markdown/src/utils.ts:66)

Optional, non-blocking suggestions

• Add one import test that pins whichever CJK behaviour is chosen. The tests export bold 你好。 with a reference, but none imports **注意：**请先登录.
• The Test plan's Before block shows MarkdownEscaping.test.ts failures from merging main into the original head rather than the new tests failing on main; AGENTS.md (Commits and Pull Requests) asks for the real failing output of the command that was run.
• Export now does more work per paragraph (joinPieces, decideReferences and spoilLinks in MarkdownExport.ts), and @lexical/markdown grows by roughly a quarter. It stays linear, but apps that export a large document on every change may notice it, so it may be worth a profile.

No public API, TypeScript or Flow declaration changes, and the module-scope additions keep the package tree-shakeable.

[meta-code-review]

With every P and S character counted as punctuation, stored CJK Markdown that main reads as bold, such as **注意：**请先登录 (which main's exporter also writes), now imports with literal ** and re-exports as \*\*注意：\*\*请先登录. That is what CommonMark 0.31.2 says; whether it ships before a CJK-friendly rule, or as a [breaking] change per AGENTS.md's Backwards Compatibility section, is a maintainer decision.

[meta-code-review]

This looks good to me. Following CommonMark's Unicode punctuation and whitespace fixes real import bugs (See –**[here](https://example.com)** now., **a.**x**`c`** reading correctly, the _ shortcut after –, “ or an emoji), the exporter's character references keep formatting through both @lexical/markdown and @lexical/mdast, and the new tests fail on main and pass here. Everything below is optional and non-blocking.

CJK content that main imports as bold (acknowledged follow-up)

Stored Markdown such as **注意：**请先登录, - **要点：**说明文字 or 请**“注意”**这里 imports as bold on main, and it is what main's exporter writes for that formatting; with this change it imports with literal ** and the next export escapes them. That is what CommonMark 0.31.2 requires (： is Unicode punctuation, so the closing ** between ： and 请 is not right-flanking), it matches GFM, markdown-it and @lexical/mdast, and the PR description and the changeset's Verify: paragraph already name it and leave a CJK-friendly rule to a follow-up, so I'm not treating it as a blocker. Two small suggestions around it:

• Add one import test that pins the current behaviour, e.g. **注意：**请先登录 → literal ** (packages/lexical-markdown/src/__tests__/unit/LexicalMarkdown.test.ts); today the tests export bold 注意： with a reference but never import the shape main wrote, so the follow-up will have a test to flip.
• AGENTS.md's Backwards Compatibility section says a change that stops existing content working ships as a [breaking] minor changeset with Migration: and Detect:. The current [fix] entry with Verify: follows CONTRIBUTING.md's rule for a fix that changes output; whether stored CJK content counts as breaking is a maintainer call, and either category is fine to merge with.

Other optional suggestions

• The Test plan's Before block shows the MarkdownEscaping.test.ts failures from merging main into the original head, not the new tests failing on main; AGENTS.md (Commits and Pull Requests) asks for the real failing output of the command that was run.
• Export now does more work per paragraph (joinPieces, decideReferences and spoilLinks in packages/lexical-markdown/src/MarkdownExport.ts): roughly twice the time for a 300-paragraph export, and the minified package grows by about a quarter. It stays linear and long-line import got much faster, so this is just worth keeping an eye on for apps that export on every change.

No public API, TypeScript or Flow declaration changes, and the new module-scope code keeps the package tree-shakeable.

[meta-code-review]

Optional: with every P and S character counted as punctuation, **注意：**请先登录 (which main imported as bold and wrote on export) now imports with literal **, as CommonMark 0.31.2 requires and as the changeset's Verify: says. An import test pinning that shape in LexicalMarkdown.test.ts would document the current behaviour and give the CJK-friendly follow-up a case to flip.