Skip to content

WIP basic markdown handling - #79

Open
arvida wants to merge 3 commits into
mainfrom
feat/markdown-document-codec
Open

arvida wants to merge 3 commits into
mainfrom
feat/markdown-document-codec

Conversation

@arvida

@arvida arvida commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

No description provided.

arvida and others added 3 commits September 18, 2026 11:07
- The reserved pattern was `__LOCALHERO_MD_[A-Z0-9_]+__`. Underscore is in
  the class and the quantifier is greedy, so adjacent placeholders with no
  text between them matched as a single token. `**`code`**` emits
  strong-open + inlineCode + strong-close back to back and collapsed into
  one.
- That weakens the guard in applyTranslation: seenTokens held the merged
  string, the per-fragment occurrence check counted against a token the
  scan never reported separately, and a translation that dropped one of a
  run of placeholders could pass verification.
- Tokens are fixed-width by construction (makePlaceholder): U + 4 digits,
  P + 4 digits, and an 8-char uppercase hex digest. Matching that grammar
  instead of a greedy class is exact.
- Occurs in real content: extracting app/content/docs/review.md yields 28
  units, and u0016 has three adjacent placeholders.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- headingPath was a sparse array indexed by depth-1, so a document that
  starts at H2 left index 0 as a hole and the context serialized a null
  ahead of the real heading. Documents that skip a level (H2 straight to
  H4) had the same gap in the middle.
- A stack popped by depth carries only the headings actually in scope, so
  the path is dense and ordered whatever level the document starts at.
- Also fixes the TS2532 build error from reading .at(-1)?.depth in a
  comparison, which is undefined on an empty stack.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant