docs(conf): replace blanket anchor-URL ignore with targeted domain exceptions - #556
Conversation
'linkcheck_ignore' contained r'.*#.*', which skipped every URL with a '#' entirely -- including the base URL reachability check, not just the anchor. That hides genuinely broken links, not just unreliable anchor checks. Replace it with targeted 'linkcheck_anchors_ignore_for_url' entries for the specific domains verified (by running linkcheck against every subproject and inspecting the raw HTML each flagged link received) to produce false "Anchor not found" positives -- sites that render their heading ids/anchors client-side with JavaScript, use client-side hash routing instead of real anchors, or serve markup that reproducibly defeats Python's HTML parser. This still checks that every URL is reachable; it only skips the unreliable in-page anchor match for these known-bad domains. Verified across the aws, azure, google, oracle, and ibm subprojects: zero anchor false positives remain, and no genuine link breakage was uncovered or masked by the change.
|
Three RTD checks are failing here ( I don't think this is caused by this PR's diff: the only change is to I also reproduced the The base commit ( @k-dimple, do you have any idea what might be causing these three RTD checks to fail? |
|
@aahil-khan thanks for the PR! The three RTD failures were some transient 502 errors. They were resolved with a rebuild. The changes look good, I tried them out locally as well. I'll go ahead and merge this. Thanks for the thorough analysis and update! |
Summary
linkcheck_ignorecontainedr'.*#.*', which skipped every URL containing a#— not just the anchor check, but the entire link, including whether the base URL was reachable.This meant a genuinely dead link could silently pass
make linkcheckas long as it contained a fragment.This replaces the blanket ignore with targeted
linkcheck_anchors_ignore_for_urlentries (a Sphinx builtin, already used here forgithub.com) for the specific domains that produce falseAnchor not foundpositives.URLs are still checked for reachability; only the unreliable in-page anchor match is skipped.
How the Domain List Was Built
I removed the blanket ignore locally and ran
make linkcheckviasphinx-build -b linkcheckagainst every subproject in the repo:For every link flagged as
[broken] ... Anchor 'x' not found, I fetched the same URL and inspected the raw HTML received by the linkcheck builder to distinguish genuine breakage from false positives.Every false positive fell into one of three categories:
Client-side rendered anchors —
ubuntu.com,docs.aws.amazon.com,developer.hashicorp.com,docs.oracle.com, andazure.microsoft.cominject heading IDs via JavaScript after page load. The static HTML fetched by Sphinx's linkcheck does not contain the target ID, even though the link works correctly for a real reader.user-content-<slug>, while the public URL fragment omits theuser-content-prefix. Client-side JavaScript bridges the gap, but a static fetch cannot.Client-side hash routing, not anchors at all —
portal.azure.com/portal.azure.cn(#create/...) andconsole.aws.amazon.com(#/case/...) use#for SPA routing. There is no server-rendered element for Sphinx to find.Flaky parsing —
cloud.ibm.com: the anchor genuinely exists in the HTML, verified with plaincurl, but I reproduced the failure twice using Sphinx's ownAnchorCheckParserdirectly against the live response. Python'shtml.parserintermittently fails to find the anchor. This is worth raising with Sphinx upstream separately and is out of scope for this PR.No genuinely broken link was found or newly masked by this change.
Verification
Ran
make linkcheckper subproject after the change:Anchor not foundresults remain.[broken]results across the repo are unrelated pre-existing issues (for example,askubuntu.comand a couple of Red Hat documentation URLs returning 403 to automated requests). These are not anchor-related and are not touched by this PR.