128128// is *followable* -- it just lands on raw MDX source), and whether an external
129129// URL is alive (that is lychee's job and it needs the network).
130130//
131+ // ## The three link shapes read, and the one that only looks like a fourth
132+ //
133+ // The header above claims "every outbound link in a PUBLISHED markdown file",
134+ // so the extractor owes that population every shape a reader can actually
135+ // follow. Three of them:
136+ //
137+ // * INLINE_LINK `[text](dest)`, ``, title optional
138+ // * REFERENCE_DEFINITION `[label]: dest` at the head of a line
139+ // * AUTOLINK a bare absolute URI wrapped in angle brackets
140+ //
141+ // The autolink was missing for this gate's first three days, and the hole was
142+ // not academic: 13 docs-site links across 6 published READMEs sat in it,
143+ // every one of them in the "Docs" / "API Reference" footer -- the most
144+ // prominent link position these files have. All 13 resolved, so nothing was
145+ // broken through it; what was broken was the population. A 404 written in that
146+ // form was read by NONE of the four assertions and shipped to npm green.
147+ //
148+ // The shape that only looks like a fourth is the pointy-bracket DESTINATION,
149+ // `[text](<dest>)` -- angle brackets INSIDE an inline link's parentheses, which
150+ // is how CommonMark spells a destination containing a space. That is not an
151+ // autolink and never was; `INLINE_LINK`'s own `(<...>|...)` alternative has
152+ // always read it. The two are distinguished POSITIONALLY here: `extractLinks`
153+ // records the span each link construct consumed and refuses an autolink match
154+ // that starts inside one. Without that, `[d](<https://objectstack.ai/docs/x>)`
155+ // -- a pointy destination that happens to contain no space, and so is a
156+ // well-formed autolink body too -- would be counted twice, and the census this
157+ // gate prints would over-report by one per occurrence. The `--self-test` pins
158+ // the count, not just the presence, for exactly that reason.
159+ //
160+ // Deliberately NOT matched: CommonMark's EMAIL autolink, the address-shaped
161+ // `<foo@example.com>` with no scheme. It renders as a `mailto:` link, so it is
162+ // a link -- but it can never be a docs-site URL, so every assertion here would
163+ // classify it `external` and read no further. Matching it would widen the
164+ // recognizer and change no verdict. The scheme'd `<mailto:foo@example.com>`
165+ // form IS matched, because it costs nothing to fold into the URI pattern, and
166+ // it too classifies `external`; a pin holds that so a later reader does not
167+ // have to re-derive which of the two is in.
168+ //
131169// ## Why fenced blocks and code spans are stripped first
132170//
133171// A markdown link inside a code fence is example text, not a link. This gate
@@ -196,6 +234,26 @@ const INLINE_LINK = /!?\[[^\]]*]\(\s*(<[^>\n]*>|[^)\s]*?)(?:\s+(?:"[^"]*"|'[^']*
196234/** `[label]: dest` at the start of a line -- the reference-definition form. */
197235const REFERENCE_DEFINITION = / ^ { 0 , 3 } \[ [ ^ \] ] + ] :\s * ( < [ ^ > \n ] * > | \S + ) / ;
198236
237+ /**
238+ * CommonMark's URI autolink: a bare absolute URI in angle brackets, rendered as
239+ * a live link by GitHub and by npm alike.
240+ *
241+ * Transcribed from the CommonMark scheme production rather than approximated: 2
242+ * to 32 characters, opening with an ASCII letter, continuing with letters,
243+ * digits, `+`, `.` or `-`; then a colon; then any run of characters that are
244+ * not ASCII control, space, or either angle bracket. The final class is what
245+ * keeps the pattern honest about the shapes it must NOT claim -- `<not a url>`
246+ * carries a space and no scheme, and an HTML tag like `<br>` has no colon, so
247+ * neither can match. Both are pinned.
248+ *
249+ * The absent `g`-flag twin of the pointy-destination question above: this one
250+ * IS global, and `extractLinks` filters its matches by span rather than by
251+ * pattern, because a pointy destination and an autolink are genuinely the same
252+ * characters in different positions -- no regex reading one line at a time can
253+ * separate them, and a lookbehind that tried would be guessing at nesting.
254+ */
255+ const AUTOLINK = / < ( [ A - Z a - z ] [ A - Z a - z 0 - 9 + . - ] { 1 , 31 } : [ ^ < > \x00 - \x20 ] * ) > / g;
256+
199257/**
200258 * Every outbound link destination in one markdown document, with the 1-based
201259 * line it was written on.
@@ -204,6 +262,13 @@ const REFERENCE_DEFINITION = /^ {0,3}\[[^\]]+]:\s*(<[^>\n]*>|\S+)/;
204262 * discrimination mechanism, and both strippers preserve newlines so the line
205263 * number points at what a human will look at.
206264 *
265+ * Autolinks are matched LAST and filtered by span, because the two bracketed
266+ * forms are the same characters in different positions: the angle brackets of
267+ * `[d](<https://x/y>)` belong to the inline link that already claimed them, and
268+ * counting them again would inflate the census by one per occurrence. So each
269+ * link construct records the range it consumed, and an autolink starting inside
270+ * one is dropped rather than re-read.
271+ *
207272 * @param {string } markdown
208273 * @returns {{ line: number, dest: string }[] }
209274 */
@@ -212,11 +277,24 @@ export function extractLinks(markdown) {
212277 const found = [ ] ;
213278 prose . split ( '\n' ) . forEach ( ( line , idx ) => {
214279 const dests = [ ] ;
280+ /** @type {[number, number][] } half-open spans already claimed on this line */
281+ const claimed = [ ] ;
215282 INLINE_LINK . lastIndex = 0 ;
216283 let m ;
217- while ( ( m = INLINE_LINK . exec ( line ) ) !== null ) dests . push ( m [ 1 ] ) ;
284+ while ( ( m = INLINE_LINK . exec ( line ) ) !== null ) {
285+ dests . push ( m [ 1 ] ) ;
286+ claimed . push ( [ m . index , m . index + m [ 0 ] . length ] ) ;
287+ }
218288 const ref = REFERENCE_DEFINITION . exec ( line ) ;
219- if ( ref ) dests . push ( ref [ 1 ] ) ;
289+ if ( ref ) {
290+ dests . push ( ref [ 1 ] ) ;
291+ claimed . push ( [ ref . index , ref . index + ref [ 0 ] . length ] ) ;
292+ }
293+ AUTOLINK . lastIndex = 0 ;
294+ while ( ( m = AUTOLINK . exec ( line ) ) !== null ) {
295+ if ( claimed . some ( ( [ start , end ] ) => m . index >= start && m . index < end ) ) continue ;
296+ dests . push ( m [ 1 ] ) ;
297+ }
220298 for ( const raw of dests ) {
221299 // CommonMark's pointy-bracket destination: `[x](<a b>)`.
222300 const dest = raw . startsWith ( '<' ) && raw . endsWith ( '>' ) ? raw . slice ( 1 , - 1 ) : raw ;
@@ -541,15 +619,26 @@ function selfTest() {
541619 '' ,
542620 '```ts' ,
543621 'const s = "[fenced](/content/docs/never.mdx)";' ,
622+ 'const t = "<https://objectstack.ai/docs/never-fenced>";' ,
544623 '```' ,
545624 '' ,
546625 'A code span `[spanned](/content/docs/never.mdx)` is not a link.' ,
626+ 'A code span `<https://objectstack.ai/docs/never-spanned>` is not one either.' ,
547627 '' ,
548628 '[ref]: /content/docs/refdef.mdx' ,
549629 '' ,
550630 'Pointy [b](</content/docs/spaced page.mdx>) form.' ,
551631 'Titled [c](/content/docs/titled.mdx "A title") form.' ,
552632 '' ,
633+ '' ,
634+ 'Bare autolink <https://objectstack.ai/docs/autolinked> in prose.' ,
635+ 'Mail <mailto:hi@objectstack.ai> too.' ,
636+ // The double-count trap: a pointy DESTINATION whose body is also a
637+ // well-formed autolink, because it happens to contain no space.
638+ 'Pointy [d](<https://objectstack.ai/docs/pointy-no-space>) form.' ,
639+ // Angle brackets that are not links at all.
640+ 'Not <not a url> and not <br> and not <a href="https://x.ai">tagged</a>.' ,
641+ 'And not the address-shaped <foo@example.com> either.' ,
553642 ] . join ( '\n' ) ;
554643 const links = extractLinks ( doc ) ;
555644 const dests = links . map ( ( l ) => l . dest ) ;
@@ -560,8 +649,48 @@ function selfTest() {
560649 ok ( 'extractLinks drops a link title' , dests . includes ( '/content/docs/titled.mdx' ) ) ;
561650 ok ( 'extractLinks reads an image destination' , dests . includes ( '/content/docs/pic.png' ) ) ;
562651 ok ( 'extractLinks reports the real line number' , links . find ( ( l ) => l . dest === '/content/docs/x.mdx' ) ?. line === 1 ) ;
563- // Five, not seven: the fenced one and the code-spanned one are the discrimination.
564- ok ( 'extractLinks finds exactly the five PROSE destinations' , links . length === 5 ) ;
652+
653+ // ---- extractLinks: the CommonMark autolink ----------------------------
654+ // The shape this gate could not see for its first three days. Every one of
655+ // the 13 it was blind to sat in a published README's "Docs" footer, so the
656+ // ACCEPT pins below are the population and the REJECT pins are the price:
657+ // widening a recognizer is only safe if what it newly claims is exactly the
658+ // links and nothing else that wears angle brackets.
659+ ok ( 'extractLinks reads a bare autolink' , dests . includes ( 'https://objectstack.ai/docs/autolinked' ) ) ;
660+ ok ( 'extractLinks reads a mailto autolink' , dests . includes ( 'mailto:hi@objectstack.ai' ) ) ;
661+ ok (
662+ 'extractLinks reports the real line number for an autolink' ,
663+ links . find ( ( l ) => l . dest === 'https://objectstack.ai/docs/autolinked' ) ?. line === 17 ,
664+ ) ;
665+ ok (
666+ 'extractLinks SKIPS an autolink inside a fenced block' ,
667+ ! dests . includes ( 'https://objectstack.ai/docs/never-fenced' ) ,
668+ ) ;
669+ ok (
670+ 'extractLinks SKIPS an autolink inside a code span' ,
671+ ! dests . includes ( 'https://objectstack.ai/docs/never-spanned' ) ,
672+ ) ;
673+ // The reject side, one pin per shape that wears the brackets without being a
674+ // link: no scheme, no colon, a tag with attributes, and the address-shaped
675+ // EMAIL autolink this recognizer deliberately leaves out (see the header).
676+ ok ( 'extractLinks does NOT match angle-bracketed prose' , ! dests . some ( ( d ) => d . includes ( 'not a url' ) ) ) ;
677+ ok ( 'extractLinks does NOT match an HTML tag' , ! dests . some ( ( d ) => d === 'br' || d . startsWith ( 'a href' ) ) ) ;
678+ ok ( 'extractLinks does NOT match a bare email autolink' , ! dests . includes ( 'foo@example.com' ) ) ;
679+ // The double-count guard, asserted as a COUNT rather than a presence: a
680+ // pointy-bracket destination with no space is a valid autolink body too, so
681+ // a span-blind recognizer reads it twice and every census this gate prints
682+ // inflates by one per occurrence. Presence alone cannot see that.
683+ ok (
684+ 'extractLinks counts a pointy-bracket destination exactly ONCE' ,
685+ dests . filter ( ( d ) => d === 'https://objectstack.ai/docs/pointy-no-space' ) . length === 1 ,
686+ ) ;
687+ ok (
688+ 'extractLinks still unwraps the pointy destination it always read' ,
689+ dests . includes ( '/content/docs/spaced page.mdx' ) ,
690+ ) ;
691+ // Eight, not thirteen: two fenced, two code-spanned, and the three
692+ // bracket-wearing non-links are the discrimination.
693+ ok ( 'extractLinks finds exactly the eight PROSE destinations' , links . length === 8 ) ;
565694
566695 // ---- classify: every bucket, both directions --------------------------
567696 ok ( 'classify: root-relative' , classify ( '/content/docs/a.mdx' ) === 'root-relative' ) ;
@@ -753,6 +882,66 @@ function selfTest() {
753882 ) ;
754883 }
755884
885+ // ---- the autolink reaches the assertions, not just the extractor ------
886+ // Extraction is necessary and not sufficient: the defect was that all four
887+ // assertions were SILENT on this shape, so each is re-observed FAILING with
888+ // the destination written as an autolink. Pinning extraction alone would
889+ // leave the gate free to read the link and then check nothing about it.
890+ // A1 is the one assertion this shape cannot reach, and that is CommonMark's
891+ // doing rather than an omission here: an autolink body must be an ABSOLUTE
892+ // URI, so a root-relative path in angle brackets is not a link on any of the
893+ // three surfaces -- GitHub and npm render `</content/docs/x.mdx>` as literal
894+ // text. Pinned in the direction that is actually true, so a later author does
895+ // not read the missing A1 pin as a hole and "fix" it by dropping the scheme
896+ // requirement -- which would start claiming every HTML tag in the tree.
897+ const auto1 = runDoc ( 'Docs: </content/docs/automation/flows.mdx>' ) ;
898+ ok (
899+ 'a root-relative path in angle brackets is NOT an autolink (no scheme)' ,
900+ auto1 . stats . links === 0 && auto1 . findings . length === 0 ,
901+ ) ;
902+ const auto2 = runDoc ( 'Docs: <https://docs.objectstack.ai/docs/automation/flows>' ) ;
903+ ok (
904+ 'A2 FAILS on an alias origin written as an autolink' ,
905+ auto2 . findings . length === 1 && auto2 . findings [ 0 ] . kind === 'non-canonical-host' ,
906+ ) ;
907+ ok ( 'A2 counts the autolink it read' , auto2 . stats [ 'non-canonical' ] === 1 ) ;
908+ const auto3 = runDoc ( 'Docs: <https://objectstack.ai/docs/automation/no-such-page>' ) ;
909+ ok (
910+ 'A3 FAILS on a dead docs page written as an autolink' ,
911+ auto3 . findings . length === 1 && auto3 . findings [ 0 ] . kind === 'dead-page' ,
912+ ) ;
913+ const auto3ok = runDoc ( 'Docs: <https://objectstack.ai/docs/automation/flows>' ) ;
914+ ok (
915+ 'A3 SILENT on a resolving autolink, and COUNTS it resolved' ,
916+ auto3ok . findings . length === 0 && auto3ok . stats . resolved === 1 ,
917+ ) ;
918+ const auto4 = runDoc (
919+ 'Docs: <https://objectstack.ai/docs/automation/flows#no-such-heading-anywhere>' ,
920+ ) ;
921+ ok (
922+ 'A4 FAILS on a dead anchor written as an autolink' ,
923+ auto4 . stats . fragments === 1 && auto4 . findings . some ( ( f ) => f . kind === 'dead-anchor' ) ,
924+ ) ;
925+ // The card's own positive control, kept as a pin: this exact input returned
926+ // [] from the shipped extractor, which is what made the hole measurable.
927+ const autoCard = runDoc ( 'Docs: <https://objectstack.ai/docs/no-such-page-anywhere>' ) ;
928+ ok (
929+ 'the reported 404-in-an-autolink is now a finding rather than silence' ,
930+ autoCard . stats . links === 1 && autoCard . findings . some ( ( f ) => f . kind === 'dead-page' ) ,
931+ ) ;
932+ // A mailto autolink is a LINK (it is counted) and is out of scope BY NAME
933+ // (it is classified external and yields nothing) -- both halves, because
934+ // "no findings" alone is also what a recognizer that never saw it produces.
935+ const autoMail = runDoc ( 'Mail: <mailto:hi@objectstack.ai>' ) ;
936+ ok (
937+ 'a mailto autolink is counted as a link and classified external' ,
938+ autoMail . stats . links === 1 && autoMail . stats . external === 1 && autoMail . findings . length === 0 ,
939+ ) ;
940+ // The non-links, at document level: an angle bracket in prose must not
941+ // manufacture a link, or the empty-scan guard above stops meaning anything.
942+ const autoNone = runDoc ( 'Text <not a url>, a tag <br>, and <foo@example.com>.' ) ;
943+ ok ( 'bracket-wearing non-links yield no links at all' , autoNone . stats . links === 0 ) ;
944+
756945 // ---- the empty-scan guard is real, not decorative ---------------------
757946 const empty = runDoc ( 'No links here at all.\n\n```ts\nconst x = 1;\n```\n' ) ;
758947 ok ( 'a document with no links yields no findings and no links' , empty . stats . links === 0 && empty . findings . length === 0 ) ;
@@ -764,10 +953,14 @@ function selfTest() {
764953 }
765954 console . log (
766955 '✓ check:published-readme-links --self-test — extraction discrimination (fence, code span,\n'
767- + ' ref-def, pointy brackets, titles), all seven classify buckets, the remedy builder and its\n'
768- + ' refusal, the canonicaliser both ways, the pageCandidates directory/index subtlety, and\n'
769- + ' all four assertions observed both FAILING and SILENT — including the host split itself:\n'
770- + ' an alias origin is a FINDING, and is still resolved and still anchor-checked.' ,
956+ + ' ref-def, pointy brackets, titles, AUTOLINKS), all seven classify buckets, the remedy\n'
957+ + ' builder and its refusal, the canonicaliser both ways, the pageCandidates directory/index\n'
958+ + ' subtlety, and all four assertions observed both FAILING and SILENT — including the host\n'
959+ + ' split itself (an alias origin is a FINDING, and is still resolved and still\n'
960+ + ' anchor-checked) and the autolink shape (assertions 2, 3 and 4 re-observed FAILING on it;\n'
961+ + ' A1 pinned UNREACHABLE through it, since an autolink body must be an absolute URI; a\n'
962+ + ' pointy-bracket destination counted exactly ONCE; and angle-bracketed non-links —\n'
963+ + ' tags, prose, bare addresses — claimed by nothing).' ,
771964 ) ;
772965}
773966
0 commit comments