fix(seo): indexing hygiene — fallback rule, sitemap, lang, clusters, hubs, OG; no offers until the price is ruled

[objectstack-fleet]

Fixes #219

Branch. This PR carries 4 commit(s) from the seat's linear stack, replayed onto main @ 1174309. The replay is patch-equal to git diff 37f1137 lin/issue-219, with byte-identical diffs. Head: a688c3a. The staged SHAs quoted below are the pre-replay commits; their content is unchanged.

Indexing hygiene from the seat's site-wide audit: the sitemap stops listing noindexed pages, English-only pricing/legal/security pages under other locales follow the fallback rule, fallback bodies declare lang="en", the templated non-English cluster pages are noindexed, pillar pages get real inbound links, thin topic hubs are noindexed and every hub gets BreadcrumbList/ItemList, redirecting internal links are gone, the root page, cache headers, Open Graph and structured data are corrected (except SoftwareApplication offers, which waits on #214), and titles stop double-branding. seo:smoke now asserts items 1–3 (and 7) as whole-build invariants, so they cannot come back.

Measured on a production build of the stack base (7f69d97, 880 pages) and of this branch (5d685cf, 880 pages), with the same audit script.

Before / after

#	Measure	Before	After
1	Sitemap URLs	870	569
1	Sitemap URLs that are noindexed (75 marketing, 110 glossary terms, 5 glossary indexes)	190	0
1	Sitemap URLs whose canonical is another page	0	0
2	Indexable pricing/privacy/terms/refunds/security pages in ja/de/es/fr/ko	25	0
2	Duplicate-title groups among indexable pages	17	4
2	Duplicate-description groups among indexable pages	4	0
3	Pages whose main element holds text in another language than declared	215 (25 indexable)	0
3	Mismatched prose blocks	1995	0
4	Indexable non-English cluster pages	35	0
5	English pages linking each English cluster page (switchers excluded)	1	125
5	Reading-path posts linking back to their cluster	0 / 160	160 / 160
6	Indexable hubs listing fewer than 3 posts (25 with one, 26 with two)	51	0
6	Hubs with BreadcrumbList / ItemList	0 / 0	131 / 131
7	Internal links that redirect (24 to the root, 8 without trailing slash)	32 on 24 pages	0
8	Root page: noindex beside meta refresh / canonical / relative hreflang	yes / none / 9	no / /en/ / 0
9	_headers with an immutable rule for /_astro/*	no	yes
10	Pages with an invalid og:locale (en, zh_Hans, ...)	879	0
10	Pages without og:image:alt / without image width+height	879 / 879	0 / 0
10	Organization nodes with @id / sameAs	0 / 0	879 / 879
10	SoftwareApplication nodes with offers	0 / 8	0 / 8 (waits on #214)
11	Indexable titles over 60 columns / descriptions over 160	268 / 266	261 / 260
11	Indexable titles naming ObjectOS twice	27	0

The four remaining duplicate-title groups are cross-locale pairs whose strings happen to be identical (the CRM hub in en and fr; three zh-Hans/zh-Hant hubs where the conversion leaves the label unchanged). They are different-language pages already paired by hreflang.

What changed, item by item

1. Sitemap. The filter in astro.config.mjs now reads each built page and lists it only when it is not noindexed and is its own canonical. One rule covers every fallback producer (marketing, glossary, pricing/legal/security, templated clusters, thin hubs, archived posts) instead of a second copy of each registry.
2. Pricing, legal, security. src/lib/fallback.ts applies the marketing fallback rule: noindex, canonical to the English URL, hreflang only for authored locales, while the language switcher still offers every locale. Authored-ness is derived rather than listed. A copy-table entry that is the English object is a fallback, and a security locale that inherits any English field is a fallback. ja is included: like de/es/fr/ko, its security copy inherits five English sections (steps, residency rows, permission layers, deployments, FAQ). The card named only de/es/fr/ko.
3. lang on fallback content. Base takes contentLocale. A fallback body gets lang="en" on the main element, and og:locale follows it. The partly translated security pages keep their translated frame, and each inherited English part is marked lang="en". The new gate found one more case: glossary fallbacks list the reader's own-locale articles inside the English body, so those links now carry their locale.
4. Templated clusters. All 35 non-English cluster pages are noindexed, canonical to the English cluster, and absent from hreflang and from llms.txt. llms.txt is a curated map and now lists only the five English cluster pages. clusterSourceLocale in src/lib/cluster-i18n.ts is the one place to change once a locale is written per cluster.
5. Pillar links. Every page's footer links the five clusters (FooterClusters.astro). Every post that a cluster lists as its reading path gets a "Reading path" box linking back. The box shows titles only, because outside English the cluster descriptions are one template.
6. Thin hubs. Hubs listing fewer than 3 posts are noindexed and left out of every hub's hreflang. They stay built and linked from the menus. Every hub gets BreadcrumbList and ItemList JSON-LD and a title of the form "CRM · Articles · ObjectOS", which also ends the collision between the Security & Governance hub and the security page. The card's other option, an intro paragraph per hub, would need authored copy for each of 131 hubs. The current intro is termDescription, one sentence per term group with the label swapped.
7. Redirecting links. SecurityPage builds its post link with postPath, which adds the trailing slash. The 24 [ObjectOS](https://www.objectos.ai) links in two posts (16 files) now point at the locale home, and zh-Hant was regenerated. No date or updated was touched.
8. Root page. It now has absolute hreflang, canonical set to /en/, and no noindex. Pairing noindex with a redirect and a canonical sends contradictory signals, and the zero-delay refresh already keeps / out of the index. The sitemap still omits /.
9. Cache. public/_headers gives /_astro/* the header Cache-Control: public, max-age=31536000, immutable. All 184 files in that folder have hashed names, and smoke asserts this too.
10. OG and structured data.
    • og:locale values are now en_US, zh_CN, zh_TW, ja_JP, de_DE, es_ES, fr_FR and ko_KR.
    • Every page has og:image:alt (the default hero's alt text, or the post title for a cover). Width and height come from the image file.
    • The Organization has @id https://www.objectos.ai/#organization, and the WebSite, SoftwareApplication and BlogPosting publishers reference it.
    • sameAs lists only https://github.com/objectstack-ai. GitHub's API reports that address as the owner, of type Organization, of the public objectstack and www.objectos.ai repositories. No social accounts were added.
    • SoftwareApplication.offers is deliberately not added. It waits on [Decision] Pricing headline: www shows $20 / $45 per AI seat "billed annually (about two months free)", docs License & Pricing shows $24 / $54 per month with no annual price — which is the source, and is the annual discount real? #214: the headline price is under a pending maintainer decision ($20/$45 billed annually on www vs $24/$54 monthly on docs), and structured data must not assert either price until that is ruled. Once [Decision] Pricing headline: www shows $20 / $45 per AI seat "billed annually (about two months free)", docs License & Pricing shows $24 / $54 per month with no annual price — which is the source, and is the annual discount real? #214 is decided, the offers can be derived from src/lib/pricing.ts rather than restated.
11. Titles.
    • Marketing titles that already contain "ObjectOS" no longer get " · ObjectOS" appended (27 indexable, 72 overall).
    • The blog index uses the home page's heading for the list ("Practical thinking on AI-native software · ObjectOS" and its translations) instead of "Articles · ObjectOS".
    • content:lint warns when a title is wider than 60 columns or a description wider than 160, with CJK characters counted double. For posts first published on or after 2026-10-06 the warning is listed per file. The existing backlog (193 titles, 204 descriptions) is one summary line, and --snippets lists every post.
    • The 95-character title warning folds into the 60-column one. The 230-character description warning stays, because it is the AGENTS.md rule.
    • Page-name titles such as "Pricing · ObjectOS" are kept on purpose.
    • Post titles are authored content and are not rewritten here.

The new gates (scripts/seo-smoke.mjs, scripts/lib/text-lang.mjs)

Smoke reads every built page once and asserts invariants over the whole build:

1. Every sitemap URL is a built page that is indexable and its own canonical.
2. Every canonical and hreflang target is indexable and self-canonical, and a page that canonicalizes elsewhere is noindexed.
3. Every block of prose in the main element is in the language its nearest lang declares.
   • CJK text is identified by script. en/de/es/fr text is identified by stopwords that belong to only one of those languages.
   • A block with too little text counts as "cannot tell". It is never counted as a pass.
   • A per-locale control fails the run if any locale was never confirmed, so the check cannot pass while measuring nothing.
4. No internal link, canonical, og:url or hreflang redirects. That covers a missing trailing slash, the root /, a .../index.html link, and any _redirects source.

It also checks the root page, _headers, the og:locale values, og:image:alt, the Organization @id/sameAs, the hub JSON-LD, and the expectations for English vs. templated clusters.

Red on the stack base, green here. These are the committed scripts run against the base build:

SEO smoke test failed (4344 issues)
  sitemap-0.xml: lists noindexed page …            190
  … judged blocks in main are not in their declared language   215 pages (25 of them pricing/legal/security)
  internal link … redirects                         9 targets on 24 pages

On this branch:

SEO smoke test passed (870 HTML pages checked; 569 sitemap URLs against 301 noindexed pages;
5424 canonical/hreflang annotations; 15654 prose blocks language-checked; 93189 internal links redirect-checked)

On the base build the hreflang invariant has nothing to report: the English pricing pages under de/... were themselves indexable, so the language check is what catches item 2 there. To prove the invariant can fail, each of five mutations was applied to a copy of this branch's build, one at a time. Each one turned smoke red with exactly its own message, and smoke went green again once the copy was restored byte-identical:

• an hreflang to /de/pricing/
• a sitemap entry for /de/pricing/
• lang="en" removed from /de/pricing/
• a link without its trailing slash
• a canonical pointing at a noindexed page

Verification

All at 5d685cf:

• pnpm check: 0 errors, 0 warnings, 0 hints (147 files).
• pnpm build: 880 pages. The build runs content-lint --published (pass, plus the snippet summary line), svg-text-fit (OK, 112 SVGs) and content-lint --dist: 478 blog pages and 93199 internal links, all resolving.
• pnpm seo:smoke: passed, with the line quoted above.
• pnpm content:lint: pass. pnpm content:lint --dist: pass. pnpm svg:lint: OK.
• Browser:
  • The footer cluster row was checked on /en/, /ja/blog/ and /de/pricing/.
  • The Reading-path box was checked on an en, a zh-Hans and a de post.
  • Widths were 1366, 1440, 1512, 1920, 390 and 360 in light, plus 1366 and 390 in dark.
  • No horizontal overflow at any width, and the screenshots were looked at.

Acceptance notes

• Live redirects were not measured. The 308 for a slashless directory URL and the root meta-refresh hop are read from the build (Cloudflare Pages' documented behaviour) because the live host is not reachable from the session.
• Thin hubs use the site's one robots form, noindex, nofollow (seat ruling on review). noindex, follow would also let crawlers follow a hub's post links, but posts are linked from the blog index and the sitemap anyway, so a second robots form is not worth adding.
• SVG covers as og:image (not changed here). 209 indexable post pages (41 en) use an SVG cover as og:image. The major share consumers do not render SVG, so those shares have no preview image. Fixing this needs raster covers or a default raster fallback; Images: 80 posts ship 1.6 MB PNG covers (mobile LCP ≈ 8 s on slow 4G), 120 marketing pages eager-load a 1.7 MB hero, and 209 posts use an SVG og:image that social networks cannot show #220 tracks it.
• The remaining fallbacks are content work. The security sections in ja/de/es/fr/ko and the non-English cluster pages are noindexed until they are translated or written. Overriding the inherited security fields in a locale's copy makes that page indexable automatically: the rule is derived, so there is no list to update.

---

Generated by Claude Code

🤖 Generated with Claude Code

https://claude.ai/code/session_01FeA1nwBz1ohH65dvffUGKr