diff --git a/CHANGELOG.md b/CHANGELOG.md index 9ce1c1b..bfd1e87 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,12 +3,49 @@ All notable changes to Micron2HTML are documented in this file. The format follows [Keep a Changelog](https://keepachangelog.com/), and this project adheres to [Semantic Versioning](https://semver.org/). -## [Unreleased] +## [1.1.0] - 2026-07-30 + +### Changed — goal correction: MeshChat parity → NomadNet parity + +The project's stated design goal was "MeshChat parity" (matching [Reticulum MeshChat](https://github.com/liamcottle/reticulum-meshchat)'s `MicronParser.js`). That target was corrected to full NomadNet parity — rendering Micron the way the real [NomadNet](https://github.com/markqvist/NomadNet) client would, in both directions, per its own [Guide.py](https://github.com/markqvist/NomadNet/blob/master/nomadnet/ui/textui/Guide.py) (the in-app spec for page authors), [MicronParser.py](https://github.com/markqvist/NomadNet/blob/master/nomadnet/ui/textui/MicronParser.py) (the reference implementation), and RNS's `MarkdownToMicron.format_table_raw` (the table-rendering algorithm), all fetched and read line-by-line for this pass. Most of what was previously justified as "MeshChat parity" turned out to already match NomadNet's own behavior too (unknown-token consumption, the field backtick requirement, heading depth limits, no line-level `B` fill, alignment tokens working anywhere inline) — those comments were relabeled without any behavior change. + +**New features, previously unimplemented:** + +- **Tables** (`` `t ``). Rendered as literal box-drawing-character ASCII art — the same visual approach real NomadNet uses — not a semantic HTML ``, for genuine visual parity. Supports markdown-style header/separator/data rows, per-column alignment, minimum column width, an optional whole-table alignment + max-width suffix (`` `tc30 ``), and escaped pipes in cell content. Column widths use `len()` rather than `wcwidth` (documented simplification — avoids a new runtime dependency). +- **Anchors** — heading auto-anchors (slugified heading text, e.g. `>Hello World` → `id="hello-world"`), explicit `` `:name `` declarations (zero-width, first-declared-wins shared namespace), `` `[label`#name] `` named jump links, and bare `` `[label`#] `` "jump to the next heading" links. `slugify_micron()` is exported from the package. + +**Real, code-level behavior changes** (both reverse previously-shipped, explicitly-tested Micron2HTML-only leniencies/extensions that have no equivalent in real NomadNet): + +- **The inline mid-line literal-block toggle is removed.** `` `=...`= `` usable anywhere within a single line was a Micron2HTML-only extension with no real NomadNet equivalent — NomadNet's `` `= `` toggle only fires on a line that is *exactly* `` `= `` and nothing else (the multi-line block form, already correct). `` `=`!not bold`=`! `` now renders "not bold" as **bolded** text (the `` `= `` tokens are silently consumed like any unrecognized token, `` `! `` toggles bold normally around them) instead of suppressing the bold. +- **Link field-specs with more than 3 backtick-separated segments now render nothing at all**, matching NomadNet exactly (`` `[label`url`a=1`b=2`c=3] `` → no `` tag, not even fallback text). This reverses a v1.0.3 fix that leniently joined the extra segments back together — that fix was solving a MeshChat-specific quirk, not real NomadNet behavior (NomadNet's own link parser sets the URL to empty and skips emitting anything when there are more than 3 components). The correct, NomadNet-native way to pass multiple fields — a single pipe-separated segment (`` `[label`url`a=1|b=2|c=3] ``) — is unaffected and unchanged. +- **Empty heading lines now emit nothing** (no row, no blank space), matching NomadNet's `parse_line()` returning `None`. Reverses a deliberate deviation made earlier in this same body of work (rendering a visible blank row) once the broader goal was clarified to be exact parity rather than "whatever reads best in isolation." + +**Partials get richer, still without shipping JS:** + +- `` `{url`refresh`fields} ``'s `refresh` and `fields` (pipe-separated, may include `pid=`) components — previously silently discarded — are now exposed as `data-refresh`, `data-fields`, and `data-pid` attributes on the rendered link, so a consuming web app can wire up its own live-refresh behavior if it wants to. `refresh` only becomes `data-refresh` if it parses as a number `>= 1`, matching NomadNet's own "0 or omitted disables it" rule. Still no live refresh happens inside this library itself — that would require shipping JS, which this pure-Python library never has. + +**Docs corrected to match:** + +- Checkbox/radio field docs now also show NomadNet's own canonical style (empty field label, visible text written after the `>`) alongside the embedded-label form. +- Link field-specs' primary documented syntax is now pipe-separated (`` `[label`url`a=1|b=2] ``), matching NomadNet's actual syntax. +- README's "Known limitations" no longer lists tables, anchors, or "alignment tags must appear at line start" (the last of those was never a real parser rule to begin with — Guide.py's line-start wording is author style advice, not something `make_output()` actually enforces; Micron2HTML already matched real NomadNet's permissive behavior there). What's left: the partials live-refresh gap, the `wcwidth` and table-shrink-formula approximations, and a rare anchor-collision edge case in the bare-hash pre-pass — see README for details. + +See [README.md](README.md) for the full corrected syntax reference. ### Fixed -- **Page-level `#!bg=`/`#!fg=` headers now reject 6-hex values.** `_parse_header_color()` previously accepted both 3-hex (`#!bg=2a2`) and 6-hex (`#!bg=112233`) forms. Only the inline `` `Fxxx ``/`` `Bxxx `` tokens were ever restricted to 3-hex; the page-header path was an inconsistent leftover. Both colour paths now use the same 3-hex-shorthand format exclusively (each nibble doubled). +- **Out-of-order tag closes produced mismatched HTML nesting.** `_close_innermost()`/`_pop_tag()` (used by `` `b ``/`` `f `` and the `` `! ``/`` `_ ``/`` `* `` toggles) searched the open-tag stack for the target type and popped it from the middle when it wasn't actually innermost, but still emitted the close tag at the *current* output position — which a browser applies LIFO, so it closed whatever tag genuinely was innermost instead. Example: `` `B777 X`f `F975`b <> `` opened a background span, then a foreground span, then tried to close the background — the emitted `` actually closed the (empty) foreground span in the DOM, leaving the background open until end-of-line and desyncing the internal bookkeeping for the rest of the run. Fixed by having `_close_innermost()` unwind: close every tag above the target (innermost-first), close the target, then reopen the unwound tags as fresh elements. The tag stack now stores each entry's opening HTML alongside its closing HTML (`(type, open_html, close_html)`, was `(type, close_html)`) so it can replay the reopens; `_pop_tag()` is gone, the `` `! ``/`` `_ ``/`` `* `` branches route through `_close_innermost()` like `` `b ``/`` `f `` always did instead of inlining their own close+pop. Reported against a downstream consumer (vscode-mu-preview) as "background color not resetting after `` `b `` token" — confirmed the bug was entirely upstream here, not in the .mu source or the downstream renderer. 4 new tests in `TestTagNesting` cover the reported repro, a bold/colour interleave, a triple-nested two-level unwind, and the already-correct non-interleaved baseline. +- **README form-field docs described broken checkbox/radio syntax.** `` ` `` and `` `<^|name|value> `` (no backtick before the closing `>`) look plausible but don't actually render as `` elements — NomadNet's own field parser requires the backtick + label, e.g. `` ` ``. [examples/showcase.mu](examples/showcase.mu) had the same broken form. Both corrected; the mandatory-backtick rule is now called out explicitly. +- **README security section mischaracterized the file-link block.** Said `file://` scheme links are blocked; the code actually blocks any resolved URL containing a `/file/` path segment (NomadNet's download-file convention) — an actual `file://` URL isn't specially handled. Wording corrected, and the default `hash://`-vs-custom-resolver distinction is now spelled out. +- **README said "pure Python 3.9+".** `pyproject.toml` has required `>=3.10` since v1.0.8; README now matches. - **Removed stale docs for the dropped `` `FTxxxxxx ``/`` `BTxxxxxx `` inline format.** The 24-bit inline extension was removed from the parser in v1.0.2, but [README.md](README.md) and [examples/showcase.mu](examples/showcase.mu) still described/demonstrated it. Docs and the example page now only show the supported 3-hex shorthand. +- **README's `` `Fxxx `` doubling example was missing the command letter.** Showed `F40 → #ff4400` (2 hex digits after a bare `F`, which doesn't match the actual 3-hex format); corrected to `` `FF40 → #ff4400 ``. + +### Added + +- **README documents `to_text()` and `--format text`.** Both existed and were tested but were never mentioned in the library/CLI usage sections. +- **README documents the partials token** (`` `{URL`refresh`fields} ``), previously present in the parser with no docs and no test coverage. +- **37 new tests**: `TestTables` (13), `TestHeadingAnchors`/`TestExplicitAnchors`/`TestAnchorLinks` (18), `TestTagNesting` (4), divider length-2 rule (2), plus rewrites of the tests covering every behavior change above (empty headings, partials data-attributes, literal-toggle removal, link field-spec strictness). Test suite: 57 → 94. ### Infrastructure diff --git a/README.md b/README.md index eb52995..6348b52 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,8 @@ A Python library and CLI tool that converts [Micron](https://github.com/markqvis Micron is the terminal markup language used by [NomadNet](https://github.com/markqvist/NomadNet) nodes. This library lets you render Micron pages in web browsers and other HTML-capable environments. +**Design goal: NomadNet parity.** The converter aims to render the same Micron source the way the real NomadNet client would — following [NomadNet's own Guide](https://github.com/markqvist/NomadNet/blob/master/nomadnet/ui/textui/Guide.py) (the in-app spec written for page authors) and [MicronParser.py](https://github.com/markqvist/NomadNet/blob/master/nomadnet/ui/textui/MicronParser.py) (the reference implementation), not any one third-party client's behavior. Where those two disagree with each other, this library follows the Guide, since that's what real page authors read and write to. See [Known limitations](#known-limitations) for syntax NomadNet supports that isn't implemented here yet. + ## Installation ```bash @@ -23,7 +25,7 @@ cd Micron2HTML pip install -e . ``` -No runtime dependencies — pure Python 3.9+. +No runtime dependencies — pure Python 3.10+. ## Library usage @@ -45,6 +47,9 @@ html = conv.convert( # Inline-only — for titles, message previews, brand strings. # Returns formatted HTML without the
wrapper. title_html = conv.convert_inline("`F4af`!My Node`!`f") + +# Plain text — strips all formatting and colours, no HTML at all. +text = conv.to_text(micron_text) ``` `convert()` returns an HTML fragment (no `` or `` wrapper). Wrap it in your own template or use the CLI for standalone pages. @@ -68,7 +73,7 @@ conv = MicronConverter(url_resolver=my_resolver) ### Default stylesheet -A MeshChat-parity stylesheet ships with the package, named to make the design intent explicit: +A dark-terminal stylesheet ships with the package — visually styled after [Reticulum MeshChat](https://github.com/liamcottle/reticulum-meshchat)'s NomadNet renderer (its filename reflects that), independent of the markup-parsing parity goal above: ```html @@ -93,6 +98,9 @@ micron-convert page.mu --fragment # Set the node hash so internal links resolve correctly micron-convert page.mu --node-hash a1b2c3d4e5f6... + +# Strip all formatting and print plain text instead of HTML +micron-convert page.mu --format text ``` ## Micron syntax @@ -106,6 +114,8 @@ micron-convert page.mu --node-hash a1b2c3d4e5f6... #!fg=aaa Set page foreground colour ``` +Deliberately 3-hex-only, matching the inline `` `Fxxx ``/`` `Bxxx `` tags below — NomadNet's own docs would technically permit a 6-hex value here too, but without a marker distinguishing the two (the way the inline tags would need a `T` prefix), a value's meaning would silently depend on its length. One fixed width, applied consistently, is safer. + ### Headings and sections ``` @@ -117,11 +127,13 @@ micron-convert page.mu --node-hash a1b2c3d4e5f6... ### Dividers ``` ---- Horizontal rule --= Double horizontal rule -- Styled divider — repeats character `x` (e.g. -* renders centred * row) +- Horizontal rule (default style) +-= Row of `=` characters +-x Styled divider — repeats character `x` (e.g. -* renders a * row) ``` +A custom divider character only takes effect when the line is *exactly* two characters — `-` followed by one more. Any other length, including `---` or `-==`, falls back to the default rule rather than repeating what follows the first `-` (this matches NomadNet's own parser, which only reads a custom divider char off a length-2 line). + ### Inline formatting ``` @@ -129,7 +141,7 @@ micron-convert page.mu --node-hash a1b2c3d4e5f6... `*text`* Italic `_text`_ Underline -`Fxxx Set foreground colour (3-hex shorthand: each digit doubled — F40 → #ff4400) +`Fxxx Set foreground colour (3-hex shorthand: each digit doubled — `FF40 → #ff4400) `f Reset foreground colour to default `Bxxx Set background colour (3-hex shorthand) @@ -155,6 +167,32 @@ micron-convert page.mu --node-hash a1b2c3d4e5f6... `[Label`hash://a1b2c3/page.mu] Node link (resolved against node_hash) ``` +Links can also submit form-field data to a node-side page: `` `[Label`url`fields] ``, where `fields` is a single pipe-separated segment — `*` (submit every field on the page), field names to include (`username|auth_token`), or literal `key=value` pairs, mixable in any order (`*|action=preview|id=42`). The parsed spec is emitted verbatim as `data-field-spec` for the consuming application to read. A link with more than 3 backtick-separated components (e.g. `` `[Label`url`a=1`b=2] ``, extra fields separated by backticks instead of pipes) renders nothing at all, matching NomadNet exactly — always use pipes for multiple fields. + +### Anchors + +``` +`:name Declare an anchor at this point (zero-width, renders nothing) +`[Label`#name] Jump to a named anchor on this page +`[Label`#] Jump to the next `>` heading after this link's position +``` + +Every heading also becomes an anchor automatically, named by slugifying its text: lowercase, runs of non-alphanumeric characters collapsed to a single hyphen, leading/trailing hyphens stripped. `` >Hello World `` → anchor `hello-world`; `` >Introduction & Setup `` → `introduction-setup`. Explicit `` `:name `` anchors and heading auto-anchors share one namespace per page — if a name collides, the first one declared wins. + +Named-anchor links (`` `[Label`#name] ``) resolve to a plain `href="#name"` and work regardless of document order, since HTML's own fragment navigation resolves against whatever element ends up with that `id`. The bare `` `[Label`#] `` form is different: it looks forward from its own position in the document for the nearest following heading and links to that — useful for "Continue ↓" buttons without naming every section. `slugify_micron()` (also exported from the package) is available standalone if you want to build your own table of contents from a Micron document. + +Cross-page anchors (linking to a named anchor on a *different* page) use NomadNet's own `anchor=name` request-field convention rather than a URL fragment, and are out of scope here — a single `convert()` call over one page's text has no way to know another page's heading layout. + +### Partials + +``` +`{URL} Partial with no auto-refresh +`{URL`refresh} Partial that re-fetches every `refresh` seconds (0 or omitted disables it) +`{URL`refresh`field1|field2|pid=x} Partial with request fields; `pid=` targets a specific partial for refresh +``` + +In real NomadNet, a partial asynchronously loads a fragment of another page and (optionally) re-fetches it in place on an interval — see NomadNet's Guide, "Partials" section. That's not something a one-shot markup→HTML conversion can reproduce without adding JavaScript, so Micron2HTML renders a plain `[live]` link to the target URL instead — but the `refresh`/`fields`/`pid` data isn't thrown away: it's exposed as `data-refresh`, `data-fields`, and `data-pid` attributes on that link, so a consuming web app can wire up its own live-refresh behaviour if it wants to. `refresh` only becomes a `data-refresh` attribute if it parses as a number `>= 1` (matching NomadNet's own "0 or omitted disables it" rule). See [Known limitations](#known-limitations). + ### Literal blocks ``` @@ -164,23 +202,49 @@ No Micron formatting is applied inside. `= ``` +Each `` `= `` must be **alone on its own line** — that's the only form NomadNet recognizes as a literal-block toggle. `` `= `` appearing mid-line, with other content around it, isn't special syntax at all; it's just consumed as an unrecognized token like any other. + +### Tables + +``` +`t +| Name | Price | Qty | +| ---- | :---: | --: | +| Apple | Free | 5 | +| Orange | Ask, nicely | 3 | +`t +``` + +Renders as literal box-drawing-character ASCII art (`┌───┬───┐` borders, padded monospace cells) inside the normal text flow — not a semantic HTML `
` — matching what real NomadNet actually shows. The first row is the header (always left-aligned); the second is a markdown-style alignment separator (`:---:` center, `---:` right, anything else left); remaining rows are data. Column width is the widest cell in that column (ignoring Micron formatting tokens for the width calculation, floor of 3 characters); cell text itself still gets parsed normally, so a colour or bold token inside a cell renders correctly. `` `t `` can take an optional alignment letter and/or max-width number right after it (e.g. `` `tc30 ``) to center/right/left-align the whole table and cap its total width — wide tables get their widest columns shrunk down to fit. Use `\|` inside a cell for a literal pipe character. + ### Form fields Fields render as disabled `` elements unless `authenticated=True` is passed to `convert()`. ``` -` Text input — name with optional default value -` Text input with character size (e.g. `<20|name`>) -` Password input (! flag) -` Checkbox (* at end pre-checks: `) -`<^|name|value> Radio button (* at end pre-selects) +` Text input — name with optional default value +` Text input with character size (e.g. `<20|name`>) +` Password input (! flag) +` Checkbox with label (* pre-checks: `) +`<^|name|value`label> Radio button with label (* pre-selects) ``` +The backtick before the closing `>` is mandatory for every field type, including checkbox and radio — `` ` `` without it is not valid Micron and renders as plain text, not an input. This matches NomadNet's own field parser, which gives up entirely when it can't find that backtick. + +NomadNet's Guide actually favors leaving the checkbox/radio field's own label empty and writing the visible label as plain text right after the `>` (e.g. `` `Label text ``) rather than embedding it inside the field as shown above. Both forms work identically — the parser accepts either — the embedded-label form here is just easier to read in isolation. + +## Known limitations + +- **Partials render as a static link only** — no actual live auto-refresh happens (the `refresh`/`fields`/`pid` data is exposed via `data-*` attributes for a consuming app to use, but no JS ships with this library). See [Partials](#partials) above. +- **Table column widths use `len()`, not `wcwidth`** — NomadNet's own table renderer consults `wcwidth` for double-width Unicode glyphs; this converter doesn't, to avoid adding a runtime dependency this library has never had. Tables with wide (e.g. CJK) characters in cells may not align columns as precisely as real NomadNet would. +- **The table width-shrink algorithm is a faithful-effort approximation**, not a byte-for-byte port of NomadNet's exact "proportionally shrink the widest columns" formula (which isn't fully specified in the reference source) — it greedily shrinks the single widest column by one character at a time until the table fits. +- **A rare anchor-collision edge case**: the bare `` `[label`#] `` "jump to next heading" link is resolved by a pre-pass that only simulates *other headings* claiming anchor slugs, not explicit `` `:name `` anchors declared during the real render. If an explicit anchor earlier in the document happens to claim the exact slug a later heading would auto-generate, that heading loses its `id` (correct first-wins behavior), but a bare-hash link could still point at `href="#name"` where `name` belongs to the earlier anchor rather than the heading — a nearby, valid target, just not the intended one. Requires a deliberate or coincidental naming collision to trigger. + ## Security All user-supplied content is HTML-escaped before output. The converter is safe to use with untrusted Micron input — XSS via markup is explicitly tested in the test suite. -External URLs are rendered as plain `` links. File download links (`file://`) are blocked. Internal NomadNet links are resolved to application-relative hrefs. +External URLs are rendered as plain `` links. Links whose resolved path contains a `/file/` segment (NomadNet's download-file convention) are blocked and rendered as `#`. Internal NomadNet links resolve to canonical `hash://` URLs by default — pass a custom `url_resolver` to produce application-relative hrefs instead (see [Custom URL resolution](#custom-url-resolution)). ## Running tests diff --git a/__init__.py b/__init__.py index be44807..64e31cd 100644 --- a/__init__.py +++ b/__init__.py @@ -1,4 +1,4 @@ -from .converter import MicronConverter, default_url_resolver, UrlResolver +from .converter import MicronConverter, default_url_resolver, UrlResolver, slugify_micron -__all__ = ["MicronConverter", "default_url_resolver", "UrlResolver"] -__version__ = "1.0.8" +__all__ = ["MicronConverter", "default_url_resolver", "UrlResolver", "slugify_micron"] +__version__ = "1.1.0" diff --git a/cli.py b/cli.py index 5926f45..a93fa9b 100644 --- a/cli.py +++ b/cli.py @@ -32,7 +32,7 @@ def _bundled_css() -> str: - """Read the bundled MeshChat-parity stylesheet that ships with the package.""" + """Read the bundled dark-terminal stylesheet that ships with the package.""" css_path = os.path.join(os.path.dirname(__file__), "micron-meshchat.css") try: with open(css_path, "r", encoding="utf-8") as fh: diff --git a/converter.py b/converter.py index 3c46c8b..94be55d 100644 --- a/converter.py +++ b/converter.py @@ -1,8 +1,20 @@ """ Micron markup → HTML converter. -Implements the full Micron specification as documented in: - https://github.com/markqvist/NomadNet/blob/master/nomadnet/ui/textui/Guide.py +Goal: NomadNet parity — render the same Micron source the way the real +NomadNet client would, not just the way any one third-party client does. +The reference sources are NomadNet's own: + - Guide.py (the in-app spec, written for page authors): + https://github.com/markqvist/NomadNet/blob/master/nomadnet/ui/textui/Guide.py + - MicronParser.py (the reference implementation): + https://github.com/markqvist/NomadNet/blob/master/nomadnet/ui/textui/MicronParser.py + +Where the two disagree (the parser accepts something the guide never +teaches), this converter follows the guide — that's what real page authors +actually write. Where NomadNet supports something this converter can't +fully reproduce (live-refreshing partials, which need JS this pure-Python +library doesn't ship), that's called out at the relevant function rather +than silently ignored — see README's "Known limitations". Default page colours match NomadNet's terminal defaults: background #000000 (black) @@ -18,6 +30,69 @@ _HEX = frozenset("0123456789abcdefABCDEF") +# --------------------------------------------------------------------------- +# Anchors +# +# Ported verbatim from NomadNet's MicronParser.py (slugify_micron / its +# strip regex), so auto-anchor slugs generated here match what real +# NomadNet would generate for the same heading text. +# --------------------------------------------------------------------------- + +_MICRON_STRIP_RE = re.compile( + r"`[FB]T[0-9a-fA-F]{6}" + r"|`[FB][0-9a-fA-F]{3}" + r"|`:[A-Za-z0-9_\-]*" + r"|`[!*_=fbacrl`<>{]" +) + +_ANCHOR_NAME_CHARS = frozenset( + "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_-" +) + + +def slugify_micron(text: Optional[str]) -> str: + """Slugify heading text into an anchor name, matching NomadNet exactly. + + Strips Micron formatting tokens first (colour, bold/italic/underline/ + reset, alignment, link/field/partial-open, and anchor-declaration + tokens), then lowercases, collapses runs of non-alphanumeric characters + into a single hyphen, and strips leading/trailing hyphens. + + ">Hello World" -> "hello-world" + ">Introduction & Setup" -> "introduction-setup" + """ + if text is None: + return "" + stripped = _MICRON_STRIP_RE.sub("", text) + return re.sub(r"[^A-Za-z0-9]+", "-", stripped).strip("-").lower() + + +# --------------------------------------------------------------------------- +# Tables +# +# NomadNet renders `t ... `t blocks as literal box-drawing-character ASCII +# art (via RNS's MarkdownToMicron.format_table_raw), not a semantic HTML +#
— that's what real NomadNet clients actually show, so that's what +# this converter builds too, for genuine visual parity. +# --------------------------------------------------------------------------- + +_TABLE_TOGGLE_RE = re.compile(r"^`t([lcr]?)(\d*)$") + +_TABLE_H, _TABLE_V = "─", "│" +_TABLE_TL, _TABLE_TR, _TABLE_BL, _TABLE_BR = "┌", "┐", "└", "┘" +_TABLE_ML, _TABLE_MR, _TABLE_TM, _TABLE_BM, _TABLE_MM = "├", "┤", "┬", "┴", "┼" + +_TABLE_MIN_COL_WIDTH = 3 + +# Strips colour/bold/italic/underline/reset tokens for width-measurement +# purposes only — a cell's *visible* width shouldn't count its formatting. +_MICRON_TOKEN_RE = re.compile( + r"`[FB]T[0-9a-fA-F]{6}" + r"|`[FB][0-9a-fA-F]{3}" + r"|`[!*_=fb]" +) + + # --------------------------------------------------------------------------- # Braille rendering # @@ -150,6 +225,13 @@ class _DocState: literal_lines: list = field(default_factory=list) doc_fg: str = "" # CSS color from #!fg= header doc_bg: str = "" # CSS color from #!bg= header + anchors: set = field(default_factory=set) # claimed anchor names, first-wins + next_heading_map: list = field(default_factory=list) # line idx -> next heading's slug + line_index: int = 0 # current line, set by convert()'s loop + table_mode: bool = False # inside a `t ... `t block + table_lines: list = field(default_factory=list) + table_align: str = "" # "", "l", "c", "r" — captured at `t open + table_max_width: int = 100 # captured at `t open @dataclass @@ -158,8 +240,7 @@ class _InlineState: bold: bool = False italic: bool = False underline: bool = False - literal: bool = False - tag_stack: list = field(default_factory=list) # list of (type_str, close_html) + tag_stack: list = field(default_factory=list) # list of (type_str, open_html, close_html) # --------------------------------------------------------------------------- @@ -209,9 +290,11 @@ def convert(self, text: str, node_hash: str = "", base_path: str = "", """ lines = text.split("\n") doc = _DocState() + doc.next_heading_map = self._compute_next_heading_map(lines) parts = [] - for line in lines: + for idx, line in enumerate(lines): + doc.line_index = idx result = self._process_line(line, node_hash, base_path, authenticated, doc) if result is not None: parts.append(result) @@ -221,6 +304,15 @@ def convert(self, text: str, node_hash: str = "", base_path: str = "", content = html.escape("\n".join(doc.literal_lines)) parts.append(f'
{content}
') + # Flush any unclosed table + if doc.table_mode and doc.table_lines: + raw_lines, align, max_w = doc.table_lines, doc.table_align, doc.table_max_width + doc.table_mode, doc.table_lines, doc.table_align, doc.table_max_width = False, [], "", 100 + rendered = self._render_table(raw_lines, align, max_w, node_hash, + base_path, authenticated, doc) + if rendered is not None: + parts.append(rendered) + body = "\n".join(parts) # Wrap with page-level colours if #!fg/#!bg headers were found @@ -271,15 +363,30 @@ def to_text(self, text: str) -> str: def _process_line(self, line: str, node_hash: str, base_path: str, authenticated: bool, doc: _DocState) -> Optional[str]: + # ---- Inside a `t ... `t table block ---- + # Mutually exclusive with doc.literal by construction (both are only + # entered from the plain fallthrough path below), so this can safely + # come first. + if doc.table_mode: + if _TABLE_TOGGLE_RE.match(line.rstrip("\r").strip()): + doc.table_mode = False + raw_lines, align, max_w = doc.table_lines, doc.table_align, doc.table_max_width + doc.table_lines, doc.table_align, doc.table_max_width = [], "", 100 + return self._render_table(raw_lines, align, max_w, + node_hash, base_path, authenticated, doc) + doc.table_lines.append(line) + return None + # ---- Inside a multi-line literal block ---- if doc.literal: if line.rstrip() == "`=": doc.literal = False content = html.escape("\n".join(doc.literal_lines)) doc.literal_lines = [] - # MeshChat parity: literal lines inherit the surrounding - # section depth's indent (its parser applies section indent - # to every line, literal or not). + # NomadNet: literal lines inherit the surrounding section + # depth's indent — MicronParser.py's parse_line() wraps + # every widget in Padding(left=left_indent(state), ...) + # whenever depth > 0, literal or not. indent = max(0, doc.section - 1) * 20 style_attr = f' style="margin-left:{indent}px"' if indent else '' return f'
{content}
' @@ -301,6 +408,16 @@ def _process_line(self, line: str, node_hash: str, base_path: str, stripped = line.rstrip("\r") + # ---- Table start: standalone `t[align][width] line ---- + m = _TABLE_TOGGLE_RE.match(stripped.strip()) + if m: + align_char, width_str = m.group(1), m.group(2) + doc.table_mode = True + doc.table_lines = [] + doc.table_align = align_char + doc.table_max_width = int(width_str) if width_str else 100 + return None + # ---- Literal block start/end: standalone `= line ---- if stripped.strip() == "`=": doc.literal = True @@ -314,16 +431,18 @@ def _process_line(self, line: str, node_hash: str, base_path: str, # ---- Section headings: line starts with one or more > ---- if line.startswith(">"): - level = 0 - while level < len(line) and line[level] == ">": - level += 1 + level, heading_text = self._split_heading(line) doc.section = level - heading_text = line[level:].strip() if not heading_text: - # MeshChat parity: empty heading line still emits a blank row - # (its parseLine returns null and the outer convertMicronToHtml - # loop appends a
). State (section depth) is updated above. - return '
' + # NomadNet: parse_line() returns None for an empty heading — + # no row at all, not even blank space. Section depth is + # still updated above. + return None + # Auto-anchor: every heading's slugified text becomes a jump + # target, claimed before parsing the text itself so a same-slug + # explicit `:name inside this same heading loses the tie. + slug = slugify_micron(heading_text) + claimed = self._claim_anchor(doc, slug) inner = self._parse_inline(heading_text, node_hash, base_path, authenticated, doc) # Heading bg extends to the container's left edge for ALL levels @@ -332,8 +451,10 @@ def _process_line(self, line: str, node_hash: str, base_path: str, # while their bg still spans the full row. text_indent = (level - 1) * 20 style_attr = f' style="padding-left:{text_indent}px"' if text_indent else '' - # MeshChat parity: only heading levels 1–3 have a bg block; level - # 4+ falls back to the "plain" style (no bg, default fg). We + id_attr = f' id="{html.escape(claimed)}"' if claimed else '' + # NomadNet: only heading levels 1-3 have a defined style + # (STYLES_DARK/STYLES_LIGHT in MicronParser.py only define + # heading1/2/3); level 4+ falls back to plain rendering. We # render levels >3 as `.mu-line` so they get plain rendering. if level == 1: cls = "mu-h1" @@ -343,33 +464,41 @@ def _process_line(self, line: str, node_hash: str, base_path: str, cls = "mu-h3" else: cls = "mu-line" - return f'
{inner}
' + return f'
{inner}
' # ---- Dividers ---- - # MeshChat parity: only lines starting with `-` produce dividers. + # NomadNet: only lines starting with `-` produce dividers. # `=-`, `==`, `===` etc. fall through and render as regular text. s = line.strip() if s and s[0] == "-": indent = max(0, doc.section - 1) * 20 style_attr = f' style="margin-left:{indent}px"' if indent else '' - if len(s) == 1: - # `-` alone — thin solid rule (browser-default
) - return f'
' - if s[1] == "=": - # `-=` — row of `=` characters - return f'
' - # `--`, `-~`, `-*`, `-X`, etc. — styled divider; preserve the - # character so the renderer can repeat it across the row. - char_content = html.escape(s[1]) - return f'
{char_content}
' + # NomadNet's parser only honours a custom divider character when + # the line is *exactly* "-" + one more character (see + # MicronParser.py's parse_line: `if len(line) == 2`). Any other + # length — a bare `-`, or `---`, `-==`, etc. — falls back to the + # default rule, regardless of what follows the first `-`. + if len(s) == 2: + if s[1] == "=": + # `-=` — row of `=` characters + return f'
' + # `-~`, `-*`, `-X`, etc. — styled divider; preserve the + # character so the renderer can repeat it across the row. + char_content = html.escape(s[1]) + return f'
{char_content}
' + # `-` alone, or any other length (`--`, `---`, `-==`, …) — + # default thin solid rule (browser-default
). + return f'
' # ---- Empty line ---- if not line.strip(): return '
' # ---- Regular text line ---- - # MeshChat parity: no line-level bg from a leading `B` token (its - # parser doesn't do that). Bg only applies inside the explicit span. + # NomadNet: `B` only sets the colour state used to style each text + # part as it's emitted (see make_style() in MicronParser.py) — there's + # no concept of a leading `B` token filling the whole row. Bg only + # applies inside the explicit span. inner = self._parse_inline(line, node_hash, base_path, authenticated, doc) style_parts = [] @@ -396,16 +525,6 @@ def _parse_inline(self, text: str, node_hash: str, base_path: str, while i < n: ch = text[i] - # ---- Inline literal mode: pass through until closing `= ---- - if state.literal: - if ch == "`" and i + 1 < n and text[i + 1] == "=": - state.literal = False - i += 2 - else: - out.append(html.escape(ch)) - i += 1 - continue - # ---- Backslash escape ---- if ch == "\\" and i + 1 < n: out.append(html.escape(text[i + 1])) @@ -429,33 +548,30 @@ def _parse_inline(self, text: str, node_hash: str, base_path: str, # Bold (`!) elif nc == "!": if state.bold: - out.append("") - self._pop_tag(state, "strong") + self._close_innermost(state, "strong", out) else: out.append("") - state.tag_stack.append(("strong", "")) + state.tag_stack.append(("strong", "", "")) state.bold = not state.bold i += 1 # Underline (`_) elif nc == "_": if state.underline: - out.append("") - self._pop_tag(state, "underline") + self._close_innermost(state, "underline", out) else: out.append('') - state.tag_stack.append(("underline", "")) + state.tag_stack.append(("underline", '', "")) state.underline = not state.underline i += 1 # Italic (`*) elif nc == "*": if state.italic: - out.append("") - self._pop_tag(state, "em") + self._close_innermost(state, "em", out) else: out.append("") - state.tag_stack.append(("em", "")) + state.tag_stack.append(("em", "", "")) state.italic = not state.italic i += 1 @@ -464,8 +580,9 @@ def _parse_inline(self, text: str, node_hash: str, base_path: str, i += 1 color, i = self._parse_color(text, i, n) if color: - out.append(f'') - state.tag_stack.append(("fg", "")) + open_html = f'' + out.append(open_html) + state.tag_stack.append(("fg", open_html, "")) # Reset foreground (`f) elif nc == "f": @@ -477,8 +594,9 @@ def _parse_inline(self, text: str, node_hash: str, base_path: str, i += 1 color, i = self._parse_color(text, i, n) if color: - out.append(f'') - state.tag_stack.append(("bg", "")) + open_html = f'' + out.append(open_html) + state.tag_stack.append(("bg", open_html, "")) # Reset background (`b) elif nc == "b": @@ -499,47 +617,77 @@ def _parse_inline(self, text: str, node_hash: str, base_path: str, doc.align = "" i += 1 - # Inline literal mode (`=) - elif nc == "=": - state.literal = True - i += 1 - # Link (`[label`URL`field1=v1`field2=v2…] or `[URL]) + # NomadNet: `[label`url`fields] has at most 3 backtick- + # separated components (fields is itself pipe-separated for + # multiple values, e.g. `a=1|b=2`). More than 3 components + # renders nothing at all — MicronParser.py's link handler + # sets link_url = "" in that case, and its `if len(link_url) + # != 0:` guard skips emitting anything. elif nc == "[": i += 1 # past [ end = text.find("]", i) if end != -1: link_inner = text[i:end] parts = link_inner.split("`") - if len(parts) >= 2: - lbl, url = parts[0], parts[1] - # Preserve all backtick-separated field specs. - # Earlier versions only took parts[2], silently - # dropping every field after the first. - fspec = "`".join(parts[2:]) if len(parts) > 2 else "" + if len(parts) == 1: + url, lbl, fspec = parts[0], "", "" + elif len(parts) == 2: + lbl, url, fspec = parts[0], parts[1], "" + elif len(parts) == 3: + lbl, url, fspec = parts[0], parts[1], parts[2] else: - url = parts[0] - lbl = "" - fspec = "" - href = self._resolve_url(url, node_hash, base_path) - display = html.escape(lbl) if lbl else html.escape(url) - extra = (f' data-field-spec="{html.escape(fspec)}"' - if fspec else "") - out.append( - f'' - f'{display}' - ) + lbl, url, fspec = "", "", "" + if url: + # `#`-prefixed URLs are page-local anchor jumps, + # not resolved through the normal URL resolver: + # named (`#name`) or "jump to the next heading + # after this point" (bare `#`). + if url == "#": + href = self._resolve_bare_hash_link(doc) + elif url.startswith("#"): + href = url + else: + href = self._resolve_url(url, node_hash, base_path) + display = html.escape(lbl) if lbl else html.escape(url) + extra = (f' data-field-spec="{html.escape(fspec)}"' + if fspec else "") + out.append( + f'' + f'{display}' + ) i = end + 1 else: out.append("[") + # Explicit anchor (`:name) — zero-width jump target. + # NomadNet: name chars are A-Za-z0-9_-, terminated by any + # other character. Shares the same claim/first-wins + # namespace as heading auto-anchors (_claim_anchor). Emitted + # as an empty inline at the exact point it + # appears — HTML fragment navigation resolves against any + # element with a matching id, not just block containers, so + # this needs no special-casing for multiple anchors on one + # line or an anchor alone on an otherwise-empty line. + elif nc == ":": + i += 1 + start = i + while i < n and text[i] in _ANCHOR_NAME_CHARS: + i += 1 + name = text[start:i] + claimed = self._claim_anchor(doc, name) + if claimed: + out.append(f'') + # Field (`) - # MeshChat parity: a field requires a backtick between - # ``. Without it, MeshChat's - # parseField returns null and the `<` is silently eaten. - # We mirror that exactly so checkbox/radio shorthand that - # omits the backtick (``, `<^|name|value>`) - # produces the same broken render in both renderers. + # NomadNet: a field requires a backtick between `` — MicronParser.py's field handler does + # `backtick_pos = line.find('`', field_start)` and gives up + # entirely if it's not found (`pass # No '`', invalid field`). + # We mirror that exactly, including for checkbox/radio + # shorthand that omits the backtick (``, + # `<^|name|value>`) — real NomadNet renders it as broken text + # too, not an input. elif nc == "<": field_start = i + 1 backtick_pos = text.find("`", field_start) @@ -550,18 +698,30 @@ def _parse_inline(self, text: str, node_hash: str, base_path: str, out.append(self._render_field(field_content, field_data, authenticated)) i = end + 1 else: - # Malformed — eat the `<` silently, matching MeshChat. + # Malformed — eat the `<` silently, matching NomadNet. i += 1 - # Dynamic include (`{URL`refresh}) + # Partial (`{URL`refresh`fields}) + # NomadNet's partials asynchronously load and (optionally) + # periodically re-fetch a fragment of another page in place — + # see Guide.py's "Partials" section. That's not something a + # one-shot markup->HTML conversion can reproduce without + # adding JS, so this renders a plain clickable link to the + # target URL instead. The `refresh` and `fields` (pipe- + # separated, may include `pid=`) components are exposed + # as data-refresh/data-fields/data-pid attributes so a + # consuming web app can wire up its own live-refresh + # behaviour if it wants to — no JS shipped here. elif nc == "{": end = text.find("}", i + 1) if end != -1: dyn_inner = text[i + 1:end] - dyn_url = dyn_inner.split("`")[0].strip() + dyn_parts = dyn_inner.split("`") + dyn_url = dyn_parts[0].strip() href = self._resolve_url(dyn_url, node_hash, base_path) + extra = self._render_partial_data_attrs(dyn_parts) out.append( - f'[live]' + f'[live]' ) i = end + 1 else: @@ -592,16 +752,21 @@ def _parse_color(self, text: str, i: int, n: int): Format: 3 hex chars — each nibble doubled (f→ff, 8→88, 0→00). - MeshChat parity: + NomadNet parity notes: - Always consume the next 3 chars after F/B (if available), - regardless of whether they're valid hex. This matches NomadNet's - MicronParser and Liam Cottle's MicronParser.js, both of which - do `line.substr(i+1,3)` + `skip = 3` unconditionally. + regardless of whether they're valid hex. Both MicronParser.py + (`line[i+1:i+4]` + `skip = 3`) and MeshChat's MicronParser.js + (`line.substr(i+1,3)` + `skip = 3`) do this unconditionally. - If the 3 chars aren't valid hex, no colour is applied (the colour-state holds the invalid string; rendering ignores it), but the 3 chars are still consumed so they don't leak as text. - - The 24-bit `T<6hex>` extension that Micron2HTML used to accept - has been dropped — neither MeshChat nor NomadNet support it. + - NomadNet's *reference parser* also accepts a `T<6hex>` 24-bit + form (`FTrrggbb`) — but its own Guide.py never teaches this to + page authors, and MeshChat doesn't implement it either. Since + the point of this converter is to render what real `.mu` pages + actually contain, not every corner of one client's source, we + deliberately stay 3-hex-only: any real-world page relying on + `FT<6hex>` wouldn't render correctly in MeshChat anyway. Returns (css_color_str | None, new_index). """ @@ -609,16 +774,26 @@ def _parse_color(self, text: str, i: int, n: int): h3 = text[i:i + 3] if all(c in _HEX for c in h3): return "#" + "".join(c * 2 for c in h3).lower(), i + 3 - # Invalid hex — still consume the 3 chars to match MeshChat, + # Invalid hex — still consume the 3 chars to match NomadNet, # but signal "no colour" so the caller doesn't open a span. return None, i + 3 return None, i def _parse_header_color(self, value: str) -> Optional[str]: - """Parse a #!fg=X or #!bg=X color value. Returns CSS color or None. - - 3-hex only, matching the inline `Fxxx`/`Bxxx` colour format — - each nibble is doubled (f -> ff, 8 -> 88, 0 -> 00). + """Parse a #!fg=X or #!bg=X page-header color value. + + 3-hex only. Guide.py's "Page Foreground and Background Colors" + section doesn't actually state a length restriction for the header + value — it just gives a 3-hex example (`#!bg=444`) — and NomadNet's + colour-rendering helpers technically accept 6-hex strings too. We + deliberately restrict to 3-hex anyway, for the same reason as the + inline `Fxxx`/`Bxxx` tags (see `_parse_color`): there's no marker + distinguishing "3-hex" from "6-hex" the way the inline tags use a + `T` prefix, so allowing both here just means a mistyped or + ambiguous value silently changes meaning depending on its length. + One fixed width, applied consistently everywhere, is safer. + + Returns CSS color string or None. """ v = value.strip() if len(v) == 3 and all(c in _HEX for c in v): @@ -629,11 +804,109 @@ def _resolve_url(self, url: str, node_hash: str, base_path: str) -> str: """Convert a Micron URL to an href via the configured resolver.""" return self._url_resolver(url, node_hash, base_path) + def _compute_next_heading_map(self, lines: list) -> list: + """For each line index, find the nearest heading strictly after it. + + Powers the bare `[label`#] link ("jump to the next heading after + this point"). Returns a list parallel to `lines`, each entry either + the anchor slug of the nearest following heading or None. + + Two passes: forward records which line has which heading's slug + (re-simulating first-wins collision handling locally, matching + `_claim_anchor`); backward fills each index from what's `upcoming` + *before* folding in that same line's own slug, so a heading never + targets itself. + """ + n = len(lines) + slug_at = [None] * n + seen = set() + for k, raw in enumerate(lines): + if raw.startswith(">"): + _, heading_text = self._split_heading(raw) + if heading_text: + slug = slugify_micron(heading_text) + if slug and slug not in seen: + seen.add(slug) + slug_at[k] = slug + + next_map = [None] * n + upcoming = None + for k in range(n - 1, -1, -1): + next_map[k] = upcoming + if slug_at[k] is not None: + upcoming = slug_at[k] + return next_map + + def _resolve_bare_hash_link(self, doc: "_DocState") -> str: + """Resolve a bare `#` link to the next heading after this point. + + Falls back to a harmless "#" when there's no following heading, or + no document context at all (e.g. convert_inline(), which never runs + the multi-line pre-pass so next_heading_map is empty). + """ + if doc.next_heading_map and doc.line_index < len(doc.next_heading_map): + slug = doc.next_heading_map[doc.line_index] + if slug: + return f"#{slug}" + return "#" + + def _split_heading(self, line: str) -> tuple: + """Split a `>`-prefixed line into (level, stripped heading text).""" + level = 0 + while level < len(line) and line[level] == ">": + level += 1 + return level, line[level:].strip() + + def _claim_anchor(self, doc: "_DocState", name: str) -> Optional[str]: + """Claim an anchor name in the document's shared namespace. + + Returns the name if it was successfully claimed (non-empty, not + already taken), else None. First declared wins — a later duplicate + (whether another heading's auto-slug or an explicit `:name) is + silently ignored, matching NomadNet's own anchor-collision rule. + """ + if not name or name in doc.anchors: + return None + doc.anchors.add(name) + return name + + def _render_partial_data_attrs(self, dyn_parts: list) -> str: + """Build data-refresh/data-fields/data-pid attributes for a partial. + + `dyn_parts` is the backtick-split `{url`refresh`fields}` content. + Matches NomadNet's own parse_partial(): refresh is a float, and a + value < 1 (including 0 or unparseable) disables refresh entirely — + not just "any positive number". `fields` is pipe-separated; a + `pid=` entry is also surfaced as its own data-pid attribute, + mirroring NomadNet's special-casing of that one field. + """ + attrs = [] + + if len(dyn_parts) > 1: + try: + refresh = float(dyn_parts[1]) + except ValueError: + refresh = None + if refresh is not None and refresh >= 1: + attrs.append(f' data-refresh="{refresh}"') + + if len(dyn_parts) > 2 and dyn_parts[2]: + fields = dyn_parts[2] + attrs.append(f' data-fields="{html.escape(fields)}"') + for f in fields.split("|"): + if f.startswith("pid="): + attrs.append(f' data-pid="{html.escape(f[len("pid="):])}"') + break + + return "".join(attrs) + def _render_field(self, field_content: str, field_data: str, authenticated: bool = False) -> str: """Render a Micron input field. - Mirrors `parseField()` in liamcottle/reticulum-meshchat MicronParser.js. + Matches the field-parsing logic in NomadNet's own MicronParser.py + (the `elif c == '<':` branch), which liamcottle/reticulum-meshchat's + MicronParser.js independently arrives at the same behavior for. Formats (the `\\`` is the required separator between flags|name and default/label): @@ -641,6 +914,13 @@ def _render_field(self, field_content: str, field_data: str, checkbox : `` radio : `<^[size]|field_name|value[|*]\\`label>` + Note: NomadNet's own Guide.py teaches a slightly different checkbox/ + radio style — leave the field's own label empty (`` ` ``) + and write the visible label as plain text after the `>`. Embedding + the label inside the field (as shown above) is equally valid — the + parser accepts non-empty `field_data` as the label either way — it's + just not the style the guide's own examples use. + `field_content` is everything between `<` and the backtick. `field_data` is everything between the backtick and `>`. """ @@ -691,22 +971,185 @@ def _render_field(self, field_content: str, field_data: str, f'value="{html.escape(field_data)}" ' f'size="{field_width}"{dis} class="mu-field">') - def _pop_tag(self, state: _InlineState, tag_type: str) -> None: - for j in range(len(state.tag_stack) - 1, -1, -1): - if state.tag_stack[j][0] == tag_type: - state.tag_stack.pop(j) - return - def _close_innermost(self, state: _InlineState, tag_type: str, out: list) -> None: + """Close the named tag, wherever it sits in the stack. + + HTML closing tags are positional/LIFO, so closing something that + isn't currently innermost can't just emit its close tag in place — + that would close whatever *is* innermost instead. Unwind: close + every entry above the target (innermost-first), close the target + itself, then reopen the unwound entries (in their original order) + as fresh elements and push them back. Same colour/bold/etc. either + side, just split across an extra pair of tags at the unwind point. + """ for j in range(len(state.tag_stack) - 1, -1, -1): if state.tag_stack[j][0] == tag_type: - out.append(state.tag_stack[j][1]) - state.tag_stack.pop(j) + above = state.tag_stack[j + 1:] + for _, _, close_html in reversed(above): + out.append(close_html) + out.append(state.tag_stack[j][2]) + del state.tag_stack[j:] + for entry in above: + out.append(entry[1]) + state.tag_stack.append(entry) return def _close_all(self, state: _InlineState) -> list: - tags = [close for _, close in reversed(state.tag_stack)] + tags = [close for _, _, close in reversed(state.tag_stack)] state.tag_stack.clear() state.bold = state.italic = state.underline = False - state.literal = False return tags + + # ------------------------------------------------------------------ + # Table rendering + # ------------------------------------------------------------------ + + def _render_table(self, raw_lines: list, align: str, max_width: int, + node_hash: str, base_path: str, authenticated: bool, + doc: _DocState) -> Optional[str]: + """Render a buffered `t ... `t block as box-drawing ASCII art. + + Ports NomadNet's own MarkdownToMicron.format_table_raw() algorithm + (from RNS's rngit util) so cell content, column widths, and + alignment match what real NomadNet would draw for the same + markdown-table input. Generated rows are fed back through + `_process_line`, so cell content like a colour token renders + correctly and a table nested under a section picks up its indent + for free — same as any other text line. + """ + if len(raw_lines) < 2: + return None + + header_cells = self._parse_table_row(raw_lines[0]) + ncols = len(header_cells) + aligns = self._parse_table_alignments(self._parse_table_row(raw_lines[1]), ncols) + + data_rows = [] + for raw in raw_lines[2:]: + cells = self._parse_table_row(raw) + cells = (cells + [""] * ncols)[:ncols] + data_rows.append(cells) + + col_widths = [_TABLE_MIN_COL_WIDTH] * ncols + for row in [header_cells] + data_rows: + for j, cell in enumerate(row): + col_widths[j] = max(col_widths[j], self._visible_width(cell)) + col_widths = self._shrink_table_widths(col_widths, max_width) + + lines = [self._table_border(col_widths, "top"), + self._table_row(header_cells, col_widths, ["left"] * ncols), + self._table_border(col_widths, "mid")] + for row in data_rows: + lines.append(self._table_row(row, col_widths, aligns)) + lines.append(self._table_border(col_widths, "bottom")) + + # Wrap the whole table in the requested alignment via a direct + # state assignment (not an injected `c/`a pseudo-line) — a bare + # alignment-only line produces no visible output of its own, which + # would just add stray empty rows above/below the table. + if align: + doc.align = {"l": "left", "c": "center", "r": "right"}[align] + + rendered = [] + for raw in lines: + out = self._process_line(raw, node_hash, base_path, authenticated, doc) + if out is not None: + rendered.append(out) + + if align: + doc.align = "" + return f'
{"".join(rendered)}
' + + def _parse_table_row(self, line: str) -> list: + """Split a markdown-table row into cells on unescaped `|`.""" + line = line.strip() + if line.startswith("|"): + line = line[1:] + if line.endswith("|"): + line = line[:-1] + + cells = [] + current = [] + escaped = False + for ch in line: + if escaped: + current.append(ch) + escaped = False + elif ch == "\\": + current.append(ch) + escaped = True + elif ch == "|": + cells.append("".join(current).strip()) + current = [] + else: + current.append(ch) + cells.append("".join(current).strip()) + return cells + + def _parse_table_alignments(self, cells: list, ncols: int) -> list: + """Parse a markdown separator row (`:---:`/`--:`/`---`) per column.""" + aligns = [] + for cell in cells: + c = cell.strip() + if c.startswith(":") and c.endswith(":"): + aligns.append("center") + elif c.endswith(":"): + aligns.append("right") + else: + aligns.append("left") + aligns += ["left"] * (ncols - len(aligns)) + return aligns[:ncols] + + def _visible_width(self, text: str) -> int: + """Character width of a cell, ignoring Micron formatting tokens. + + NomadNet's own implementation also consults `wcwidth` for + double-width glyphs; deliberately not ported here (would add a + runtime dependency this pure-Python library has never had) — + documented simplification in the README. + """ + return len(_MICRON_TOKEN_RE.sub("", text)) + + def _pad_cell(self, text: str, width: int, align: str) -> str: + """Pad (or, if needed, truncate) a cell to `width` visible columns.""" + visible = self._visible_width(text) + if visible > width: + # Truncating mid-token could leave an unclosed span/strong tag — + # dropping formatting on truncation is strictly safer than that. + text = _MICRON_TOKEN_RE.sub("", text)[:width] + visible = len(text) + pad = width - visible + if align == "right": + return " " * pad + text + if align == "center": + left = pad // 2 + return " " * left + text + " " * (pad - left) + return text + " " * pad + + def _shrink_table_widths(self, col_widths: list, max_width: int) -> list: + """Greedily shrink the widest column until the table fits max_width. + + Faithful-effort port of NomadNet's "proportionally shrink the + widest columns" — not a byte-for-byte match of its exact formula, + which isn't fully specified in the reference source. + """ + widths = list(col_widths) + ncols = len(widths) + total = sum(widths) + ncols * 3 + 1 + while total > max_width and max(widths) > _TABLE_MIN_COL_WIDTH: + j = widths.index(max(widths)) + widths[j] -= 1 + total -= 1 + return widths + + def _table_border(self, col_widths: list, kind: str) -> str: + left, mid, right = { + "top": (_TABLE_TL, _TABLE_TM, _TABLE_TR), + "mid": (_TABLE_ML, _TABLE_MM, _TABLE_MR), + "bottom": (_TABLE_BL, _TABLE_BM, _TABLE_BR), + }[kind] + return left + mid.join(_TABLE_H * (w + 2) for w in col_widths) + right + + def _table_row(self, cells: list, col_widths: list, aligns: list) -> str: + padded = [self._pad_cell(c, w, a) for c, w, a in zip(cells, col_widths, aligns)] + return _TABLE_V + " " + f" {_TABLE_V} ".join(padded) + " " + _TABLE_V diff --git a/examples/showcase.mu b/examples/showcase.mu index 006d8bf..8844418 100644 --- a/examples/showcase.mu +++ b/examples/showcase.mu @@ -57,6 +57,19 @@ URL only: `[`https://example.com] -~ +>Anchors + +`[Jump to the anchor below`#custom-anchor] +`[Jump to the next heading`#] + +`:custom-anchor +This paragraph is a jump target declared with an explicit anchor tag. + +Every heading (including this one) is also an anchor automatically — its +slug is derived from the heading text. + +-~ + >Literal block `= @@ -67,12 +80,23 @@ URL only: `[`https://example.com] -~ +>Tables + +`t +| Name | Price | Qty | +| ---- | :---: | --: | +| `F3a3Apple`f | Free | `!5`! | +| Orange | Ask, nicely | 3 | +`t + +-~ + >Form fields Username: `<20|username`> Password: `<20!|password`> - Subscribe: ` Yes, sign me up - Plan: `<^|plan|free|*> Free `<^|plan|pro> Pro + Subscribe: ` + Plan: `<^|plan|free|*`Free> `<^|plan|pro`Pro> -= diff --git a/micron-meshchat.css b/micron-meshchat.css index e206a31..8283897 100644 --- a/micron-meshchat.css +++ b/micron-meshchat.css @@ -197,6 +197,23 @@ strong { font-style: italic; } +/* ---- Explicit anchors (`:name) ----------------------------------------- + * Zero-width jump target — defensive-only, an empty shouldn't take + * width by default, but this is cheap insurance against monospace + * column-alignment drift from any browser quirk. */ +.mu-anchor { + display: inline-block; + width: 0; + height: 0; + overflow: hidden; +} + +/* ---- Tables (`t ... `t) ------------------------------------------------- + * Rendered as box-drawing-character ASCII art, same as real NomadNet — the + * horizontal alignment is already baked into the padded cell text itself, + * so this class is just a marker/CSS hook, not load-bearing for layout. */ +.mu-table { display: block; } + /* ---- Literal blocks (`= ... `=) -------------------------------------- * Plain pre-styled text — no border, no bg, no padding. MeshChat doesn't * box literal content; it flows inline at the same font/colour as the diff --git a/pyproject.toml b/pyproject.toml index 61417be..2c7bdf0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "Micron2HTML" -version = "1.0.8" +version = "1.1.0" description = "Convert Micron markup (NomadNet) to HTML" readme = "README.md" license = { text = "MIT" } diff --git a/tests/test_converter.py b/tests/test_converter.py index 6e730d4..deb19eb 100644 --- a/tests/test_converter.py +++ b/tests/test_converter.py @@ -43,7 +43,7 @@ def test_h3(self, conv): assert 'class="mu-h3"' in out def test_heading_level_4_falls_back_to_mu_line(self, conv): - # MeshChat parity: only heading1/2/3 styles exist; deeper levels + # NomadNet: only heading1/2/3 styles exist; deeper levels # fall back to "plain" rendering (no bg block). out = conv.convert(">>>> Level 4") assert 'mu-line' in out @@ -55,11 +55,12 @@ def test_heading_content_parsed(self, conv): assert 'class="mu-h1"' in out assert "" in out - def test_empty_heading_emits_blank_line(self, conv): - # MeshChat parity: empty heading produces a blank row (its parseLine - # returns null and the outer loop appends a
). + def test_empty_heading_emits_nothing(self, conv): + # NomadNet: parse_line() returns None for an empty heading — no + # row, no blank space. out = conv.convert(">") - assert 'mu-blank' in out + assert 'mu-blank' not in out + assert out.strip() == "" # --------------------------------------------------------------------------- @@ -80,8 +81,26 @@ def test_double_equals(self, conv): out = conv.convert("-=") assert "mu-hr-double" in out + def test_triple_dash_falls_back_to_default_rule(self, conv): + # NomadNet only honours a custom divider character when the line is + # exactly "-" + one more character (MicronParser.py's parse_line: + # `if len(line) == 2`). "---" is length 3, so it falls back to the + # default rule rather than being treated as a "-"-repeat divider. + out = conv.convert("---") + assert "mu-divider" not in out + assert "mu-hr-double" not in out + assert "" not in out def test_unknown_token_eaten_silently(self, conv): - # MeshChat parity: backtick + unknown char are silently consumed - # (NomadNet's `default: break;` behaviour). The rendered text - # content should have "foo bar" (with the `> consumed) and no - # leftover backtick or > glyph. + # NomadNet: an unrecognized char after a backtick falls through + # make_output()'s if/elif chain with no output — same outcome as + # MeshChat's `default: break;`. The rendered text content should + # have "foo bar" (with the `> consumed) and no leftover backtick + # or > glyph. import re out = conv.convert("foo `> bar") # strip all HTML tags to get just the rendered text @@ -143,6 +163,63 @@ def test_unknown_token_eaten_silently(self, conv): assert ">" not in text +# --------------------------------------------------------------------------- +# Tag nesting (out-of-order close / unwind-reopen) +# +# HTML closing tags are positional/LIFO. Closing a non-innermost open tag +# (e.g. `b while `F is still open) can't just emit its close tag in place — +# that would close whatever's actually innermost instead. These tests lock +# in the unwind (close innermost-first down to the target, then reopen the +# unwound tags) that keeps HTML nesting valid regardless of stack order. +# --------------------------------------------------------------------------- + +class TestTagNesting: + def _assert_balanced(self, out): + import re + stack = [] + for m in re.finditer(r']*>', out): + tag = m.group(0) + if tag.startswith(" in place (which would close fg, + # not bg, in the real DOM). + out = conv.convert("`B777 X`f `F975`b <>`F333", authenticated=True) + self._assert_balanced(out) + # bg's own close must appear before "<>" is reached, not float to EOL + bg_close_pos = out.index("") # fg empty-close + bg close + after_pos = out.index("<>") + assert bg_close_pos < after_pos + # the reopened fg span must actually wrap "<>" so it's still coloured + assert ' <>' in out + + def test_bold_close_while_fg_still_open_unwinds_correctly(self, conv): + out = conv.convert("`!X`F975 Y`!Z`f", authenticated=True) + self._assert_balanced(out) + # bold must close before "Z" is emitted — not swallow it. + assert out.index("") < out.index("Z") + + def test_triple_nested_outermost_close_unwinds_two_levels(self, conv): + out = conv.convert("`!`_`*deep`!more`_more2`*end", authenticated=True) + self._assert_balanced(out) + assert "deep" in out + + def test_isolated_close_without_interleaving_still_works(self, conv): + # Baseline: closing the genuinely-innermost tag needs no unwind at + # all — must keep working exactly as before. + out = conv.convert("`B777 X`f`b <>", authenticated=True) + self._assert_balanced(out) + assert out.count("") == 1 + + # --------------------------------------------------------------------------- # Colors # --------------------------------------------------------------------------- @@ -158,17 +235,19 @@ def test_bg_3digit(self, conv): assert "background-color:#333333" in out def test_invalid_hex_consumes_three_chars(self, conv): - # MeshChat parity: F/B always consume 3 chars after the prefix - # regardless of validity. Invalid hex applies no colour but still - # eats the 3 chars so they don't leak as text. + # NomadNet: F/B always consume 3 chars after the prefix regardless + # of validity. Invalid hex applies no colour but still eats the + # 3 chars so they don't leak as text. out = conv.convert("`Fxxx hello`f") assert "xxx" not in out assert "hello" in out def test_24bit_T_format_not_supported(self, conv): - # MeshChat doesn't support `FT<6hex>` 24-bit colour. v1.0.2 drops it - # for parity. The `T` plus 2 next chars are consumed as a (failed) - # 3-char hex, leaving the tail as visible text. + # NomadNet's own reference parser DOES accept `FT<6hex>`, but its + # Guide.py never teaches it and MeshChat doesn't implement it — we + # stay 3-hex-only to match what real page authors actually write. + # The `T` plus 2 next chars are consumed as a (failed) 3-char hex, + # leaving the tail as visible text. out = conv.convert("`FT8b4513 brown`f") assert "color:#8b4513" not in out assert "color:#" not in out # no colour applied @@ -180,15 +259,18 @@ def test_header_fg_bg_3digit(self, conv): assert "color:#aaaaaa" in out def test_header_fg_bg_6digit_not_supported(self, conv): - # Page-level headers use the same 3-hex shorthand as inline colours — - # 6-hex values are invalid and silently produce no colour. + # Deliberately 3-hex-only, same as the inline colour tags: with no + # marker distinguishing 3-hex from 6-hex, allowing both would make + # a value's meaning depend silently on its length. NomadNet's own + # docs would technically permit 6-hex here, but we don't chase it. out = conv.convert("#!bg=112233\n#!fg=aabbcc\nhello") assert "background-color" not in out assert "color:#" not in out def test_line_level_bg_not_applied_to_div(self, conv): - # MeshChat parity: a leading B token doesn't fill the entire line — - # only the explicit span gets the background. + # NomadNet: `B` only sets colour state used per text part as it's + # emitted — a leading B token doesn't fill the entire line, only + # the explicit span gets the background. out = conv.convert("`B400 red `b end") # the .mu-line wrapper itself should NOT carry background-color assert 'class="mu-line"' in out @@ -278,15 +360,238 @@ def test_link_single_field_spec(self, conv): ) assert 'data-field-spec="action=register"' in out - def test_link_multi_field_spec_preserves_all(self, conv): - # Regression: earlier versions only captured parts[2] and dropped - # every field after the first. The full backtick-separated spec - # must be preserved so the renderer can forward each key=value. + def test_link_multi_field_spec_uses_pipes_not_extra_backticks(self, conv): + # NomadNet-native form: multiple fields go in ONE backtick-delimited + # segment, pipe-separated. + out = conv.convert( + "`[Go`:/page/x.mu`a=1|b=2|c=3]", + node_hash="deadbeef", + ) + assert 'data-field-spec="a=1|b=2|c=3"' in out + + def test_link_with_more_than_three_backtick_segments_renders_nothing(self, conv): + # NomadNet: more than 3 backtick-separated components makes the + # whole link vanish (link_url = ""), not a leniently-joined spec. + # This reverses a v1.0.3 fix that was solving a MeshChat-specific + # leniency, not real NomadNet behavior. out = conv.convert( "`[Go`:/page/x.mu`a=1`b=2`c=3]", node_hash="deadbeef", ) - assert 'data-field-spec="a=1`b=2`c=3"' in out + assert " Hello World") + assert 'id="hello-world"' in out + + def test_heading_auto_anchor_strips_formatting_tokens(self, conv): + out = conv.convert("> `!Bold`! Heading") + assert 'id="bold-heading"' in out + + def test_heading_auto_anchor_ampersand_and_punctuation(self, conv): + out = conv.convert("> Introduction & Setup") + assert 'id="introduction-setup"' in out + + def test_heading_level_4_also_gets_auto_anchor(self, conv): + out = conv.convert(">>>> Deep Heading") + assert 'id="deep-heading"' in out + + def test_heading_auto_anchor_collision_first_wins(self, conv): + out = conv.convert("> Same\n> Same") + assert out.count('id="same"') == 1 + + def test_empty_heading_no_anchor(self, conv): + out = conv.convert("> ") + assert 'id="' not in out + + +class TestExplicitAnchors: + def test_explicit_anchor_zero_width(self, conv): + out = conv.convert("`:mark hello") + assert 'id="mark"' in out + assert 'class="mu-anchor"' in out + assert "hello" in out + + def test_explicit_anchor_name_terminates_at_delimiter(self, conv): + out = conv.convert("`:foo-bar baz") + assert 'id="foo-bar"' in out + + def test_explicit_anchor_alone_on_empty_line(self, conv): + out = conv.convert("`:onlyanchor") + assert 'id="onlyanchor"' in out + + def test_explicit_and_heading_anchors_share_namespace(self, conv): + out = conv.convert("`:shared marker\n> Shared") + assert out.count('id="shared"') == 1 + + def test_multiple_explicit_anchors_on_one_line(self, conv): + out = conv.convert("`:first x `:second y") + assert 'id="first"' in out + assert 'id="second"' in out + + def test_explicit_anchor_name_stops_before_special_chars(self, conv): + out = conv.convert("`:foo") + assert 'id="foo"' in out + assert "