Summary
lex() knows where every argument of a directive starts and ends, and which regions it treated as literal text. Its results keep neither, so tools that edit or render directive source re-lex it themselves.
- Argument spans.
Directive has start/end for the whole directive, and decoded positional/params, but no span for each argument as written. ForLegalAI/PactTrack needs them in two places:
- Renaming an identifier (a section or definition id) in every
{{ref:}}/{{term:}} means splicing a new positional value. _positional_end in app/routes/pages.py re-scans the source to find it.
- Assembly §15.7.2 step 5 keeps a placeholder's
note "exactly as written". _argument_spans in app/services/assembly.py is a second argument lexer, copying _lex_arguments/_lex_value.
- Literal regions.
lex() blanks fenced code, code spans and HTML comments in Lexed.view but doesn't return where they are. PactTrack's export_inline._literal_regions/_fenced_code rebuild them by diffing text against view and re-matching backtick runs, to render those regions verbatim.
Proposal
Directive.positional_span: tuple[int, int] | None and Directive.param_spans: dict[str, tuple[int, int]]. The spans cover the value as written, quotes included, as offsets into the lexed text.
Lexed.literals: list[tuple[str, int, int]], holding (kind, start, end) with kind being fence, code, comment or escape.
- Export
closes_fence publicly next to FENCE_OPEN_RE/fence_end. PactTrack's closing-fence checks duplicate it.
Related: #27 (source positions for sections, blocks and diagnostics) and #28 (comment pairing in lex()).
Summary
lex()knows where every argument of a directive starts and ends, and which regions it treated as literal text. Its results keep neither, so tools that edit or render directive source re-lex it themselves.Directivehasstart/endfor the whole directive, and decodedpositional/params, but no span for each argument as written. ForLegalAI/PactTrack needs them in two places:{{ref:}}/{{term:}}means splicing a new positional value._positional_endinapp/routes/pages.pyre-scans the source to find it.note"exactly as written"._argument_spansinapp/services/assembly.pyis a second argument lexer, copying_lex_arguments/_lex_value.lex()blanks fenced code, code spans and HTML comments inLexed.viewbut doesn't return where they are. PactTrack'sexport_inline._literal_regions/_fenced_coderebuild them by diffingtextagainstviewand re-matching backtick runs, to render those regions verbatim.Proposal
Directive.positional_span: tuple[int, int] | NoneandDirective.param_spans: dict[str, tuple[int, int]]. The spans cover the value as written, quotes included, as offsets into the lexed text.Lexed.literals: list[tuple[str, int, int]], holding(kind, start, end)with kind beingfence,code,commentorescape.closes_fencepublicly next toFENCE_OPEN_RE/fence_end. PactTrack's closing-fence checks duplicate it.Related: #27 (source positions for sections, blocks and diagnostics) and #28 (comment pairing in
lex()).