Indexing hygiene: 190 noindex pages in the sitemap, 20 English pricing/legal pages indexed under other locales, ~35 templated non-en cluster pages indexed, 40 pillar pages with one inbound link

Found by the seat's site-wide audit of www.objectos.ai on 2026-10-06, run over the 871 pages of a production build of main @ 53764e0. Priority: P1. This is related to #11, but it is new work.

Findings

1. Noindex pages in the sitemap

190 pages are marked noindex, nofollow and still listed in sitemap-0.xml: 75 marketing pages, 110 glossary terms and 5 glossary indexes, in de/es/fr/ja/ko. The seat confirmed ja/platform/.
Search Console will report "Submitted URL marked noindex".
Cause: the sitemap filter in astro.config.mjs:20 excludes only /.
Fix: exclude the fallback locale/slug pairs, using the same registries seo-smoke uses.

2. English pages indexed as other locales

Pricing, privacy, terms and refunds in de/es/fr/ja/ko (20 pages) render an English body under <html lang="xx">.
They are self-canonical and listed in hreflang. The result is 6 duplicate-title groups and 4 duplicate-description groups.
Security in de/es/fr/ko serves mixed English bodies the same way.
Cause: src/lib/pricing.ts:217-226 and src/lib/legal.ts:815-845 map ja: pricingEn and so on. src/pages/[lang]/{pricing,privacy,terms,refunds}.astro:12-16 emit all 8 locales as hreflang alternates.
Fix: apply the marketing-page fallback rule: noindex, canonical to /en/, and hreflang only for authored locales. Set lang="en" on English content.

3. Wrong lang on fallback content

On the 190 fallback pages plus the 20 above, English content sits under lang="de", lang="ja" and so on, with no lang="en" on <main>. This fails WCAG 3.1.1 and 3.1.2.
Cause: Base.astro:95,144.
Fix: pass the content locale to the layout.

4. Templated non-en cluster pages

About 35 indexed non-en cluster pages (zh-Hans, ja, de, es, fr, ko, plus generated zh-Hant) share one body with the cluster name swapped (src/lib/cluster-i18n.ts:103+).
This is the scaled-content pattern AGENTS.md rule 9 warns about.
Fix: noindex them, canonical to /en/, until they are written per cluster.

5. Near-orphaned pillar pages

The 40 cluster pages each get exactly one in-site link from their own locale, the home page.
/en/ai-native-app-platform/ has 8 inbound links in total, and 7 of those are language switchers.
Fix: link the clusters from the header or footer, and link back to the cluster from every post the cluster lists as its reading path.

6. Thin topic hubs

25 hubs list a single post and 26 list two (e.g. en/blog/topics/portals/), all indexable.
Fix: noindex hubs with fewer than 3 posts, or give each an intro paragraph. Add a BreadcrumbList and an ItemList.

7. Redirecting internal links

8 security pages link to a post without the trailing slash, which costs a 308 (SecurityPage.astro:17).
16 posts link the bare https://www.objectos.ai, which goes through the root meta refresh: content/blog/business-app-in-16k-tokens/index*.mdx:81 and forward-deployed-engineer-tools.

8. Root page

src/pages/index.astro:14-24,71 combines noindex with a meta refresh, uses relative hreflang URLs, and has no canonical.

9. Cache headers

There is no public/_headers, so the hashed /_astro/* files are served with max-age=0.
Fix: Cache-Control: public, max-age=31536000, immutable for /_astro/*.

10. Structured data and Open Graph

The og:locale values en and zh_Hans are invalid (Base.astro:130); they should be en_US, zh_CN and so on.
There is no og:image:alt, width or height.
The Organization has no @id or sameAs.
SoftwareApplication has no offers.

11. Titles and descriptions

268 indexable titles are longer than 60 characters, and 266 descriptions are longer than 160. The lint warns only at 95 and 230 (scripts/content-lint.mjs:921,930).
Fix: add a 60/160 warning (measured by width for CJK), and fix the generic or double-branded titles: 54 generic, such as "Articles · ObjectOS", and 27 that say "ObjectOS" twice.

Done looks like

Each item fixed, or answered in the PR body.
seo:smoke extended so items 1–3 cannot come back:
- no noindex URL in the sitemap;
- no indexable page whose body language differs from its lang;
- no internal link that redirects.
Before/after counts in the PR body.
Gates green.

Surface

astro.config.mjs, src/layouts/Base.astro, src/pages/index.astro, src/pages/[lang]/{pricing,privacy,terms,refunds,security}.astro, src/pages/[lang]/blog/topics/**, src/lib/{pricing,legal,security,cluster-i18n,clusters}.ts, src/components/{SecurityPage,Header}.astro and the footer, public/_headers, scripts/{seo-smoke,content-lint}.mjs, and the two post link fixes.

Order

After the accessibility card.

Generated by Claude Code

Claim: PM loop, repo:www.objectos.ai seat
Session: session_01FeA1nwBz1ohH65dvffUGKr
Account: hotlong (the seat's linked user, as GET /user answers it; always the card's assignee)
Branch: claude/pm-dispatch-objectos-ju9td1, the session's one designated push branch for this repo. It carries PR #212 until its Thu 2026-10-08 slot. After that, wave 1 goes in this order: #216, #213, #217, #221, then the #209 post sweep. All five are staged and reviewed.
This card is staged in wave 2. Its work is built and verified on a local branch cut from the local integration branch stack/www-wave1 (origin/main with the five wave-1 branches merged in queue order, no conflicts). At push time only this card's own commits are replayed onto origin/main (rebase --onto).
Worktree: www-issue-219
Domain: n/a (single-lane repo)
Seat: repo:www.objectos.ai (objectstack-ai/objectstack#12224)
File surface: as the card's Surface section names. Overlaps with the other wave-2 cards (#218, #219, #224, #225) stay in separate hunks.
Stop on breach and explain in the report.
Container & model: M, mode:subagent, model: default.
Clause-②: no
Thread-read: none
Serial constraints cleared: no. Pushes are serial on the one branch, in the queue order on #209 6010716407, with wave 1 first.

Generated by Claude Code

os-dev-report

{
  "issue": 219,
  "status": "staged",
  "branch": "staged/issue-219",
  "worktree": "/home/user/www-issue-219",
  "base_sha": "7f69d97df5e82a4fb9098693142169f4c41860df (stack/www-wave1)",
  "final_commit": "1a47c50",
  "commits": [
    "5f6a3c1 content(blog): link the locale home, not the bare www root, from two posts",
    "8ec4f6c fix(seo): indexing hygiene: fallback rule, sitemap, lang, clusters, hubs, OG",
    "1a47c50 test(seo): seo-smoke guards the sitemap, body language and redirects; lint warns at 60/160"
  ],
  "pr": null,
  "pr_body_draft": "/tmp/claude-0/-home-user/2224823d-3484-535e-94a5-c071ba6af395/scratchpad/issue-219/pr-body.md (first line: Fixes #219)",
  "session": "session_01FeA1nwBz1ohH65dvffUGKr (subagent; parent's id)",
  "premise_still_valid": true,
  "summary": "All 11 items are implemented on staged/issue-219, committed in 3 commits that contain only this card's changes, so they replay cleanly with rebase --onto. Highlights: (1) the sitemap filter reads each built page and lists only indexable, self-canonical pages; (2,3) src/lib/fallback.ts applies the fallback rule to pricing, legal and security, and Base takes contentLocale, which puts lang=en on the main element of fallback bodies; (4) the 35 templated non-en cluster pages are noindexed and canonical to /en/; (5) every footer links the clusters, and the 160 reading-path posts link back; (6) hubs with fewer than 3 posts are noindexed, and every hub gets BreadcrumbList and ItemList; (7) the 32 redirecting links are fixed; (8) the root page has an absolute canonical and absolute hreflang; (9) public/_headers is added; (10) og:locale, og:image alt and size, Organization @id/sameAs and offers are fixed; (11) double-branded and generic titles are fixed, and a 60/160 column lint warning is added. seo-smoke now asserts the sitemap, canonical/hreflang, body-language and redirect invariants over the whole build. Premises: all held on the stack base, but the card's line numbers came from main @53764e0 (content-lint 95/230 is now at :1119/:1131). Item 2's security list omitted ja, which inherits the same five English sections and is covered here.",
  "per_item_before_after": {
    "1_sitemap_urls": "870 → 569",
    "1_sitemap_noindexed_urls": "190 (75 marketing, 110 glossary terms, 5 glossary indexes) → 0",
    "1_sitemap_non_self_canonical": "0 → 0",
    "2_indexable_pricing_legal_security_in_ja_de_es_fr_ko": "25 (20 pricing/legal + 5 security) → 0",
    "2_duplicate_title_groups_indexable": "17 → 4 (all cross-locale identical strings already paired by hreflang)",
    "2_duplicate_description_groups_indexable": "4 → 0",
    "3_pages_body_language_differs_from_declared": "215 (25 indexable), 1995 blocks → 0",
    "4_indexable_non_en_cluster_pages": "35 → 0",
    "5_en_pages_linking_each_en_cluster_excluding_switchers": "1 → 125",
    "5_reading_path_posts_linking_back": "0/160 → 160/160",
    "6_indexable_hubs_under_3_posts": "51 (25 with 1 post, 26 with 2) → 0",
    "6_hubs_with_BreadcrumbList_and_ItemList": "0/131 → 131/131",
    "7_redirecting_internal_links": "32 on 24 pages (24 bare-root, 8 missing slash) → 0",
    "8_root_page": "noindex+refresh, no canonical, 9 relative hreflang → no noindex, canonical https://www.objectos.ai/en/, 0 relative",
    "9_headers_immutable_astro": "absent → present (184/184 _astro files hashed)",
    "10_invalid_og_locale_pages": "879 → 0",
    "10_pages_missing_og_image_alt_or_size": "879/879 → 0/0",
    "10_organization_id_sameAs": "0/879 → 879/879",
    "10_softwareapplication_offers": "0/8 → 8/8",
    "11_indexable_titles_over_60_cols_descriptions_over_160": "268/266 → 261/260 (rest are authored post titles; lint now warns)",
    "11_double_branded_indexable_titles": "27 → 0 (72 → 0 incl. noindexed)"
  },
  "tests": "All at HEAD 1a47c50, using the committed scripts. pnpm check: Result (147 files) 0 errors / 0 warnings / 0 hints. pnpm build: 880 page(s) built; content-lint --published passed; svg-text-fit OK (112 SVGs); content lint --dist passed (478 blog pages, 93199 internal links); lock VERDICT command-exit 0. pnpm seo:smoke exit 0: SEO smoke test passed (870 HTML pages checked; 569 sitemap URLs against 301 noindexed pages; 5424 canonical/hreflang annotations; 15654 prose blocks language-checked; 93189 internal links redirect-checked). pnpm content:lint exit 0 (339 files, 44 glossary terms checked); content:lint --dist exit 0; svg:lint exit 0. RED on the stack base build (7f69d97) with the same committed smoke: exit 1, 4344 issues, including 190 x 'lists noindexed page', 215 x 'judged blocks in main are not in their declared language' (25 of them pricing/legal/security) and 9 x redirecting internal-link targets on 24 pages. On base the hreflang invariant has nothing to fire on, because the de pricing pages were wrongly indexable, so item 2 is caught there by the language check. Mutation proofs on a copy of this branch's dist (source and real dist untouched; copy restored and diff -rq identical): control green; hreflang to /de/pricing/ gave 1 issue (the expected line); sitemap entry for /de/pricing/ gave 1; lang=en removed from /de/pricing/ main gave 1; slashless link on de/security gave 1; canonical to a noindexed page gave the expected line plus a 24-line cascade; after restoring all five, green. Browser: preview server PID started and stopped by me; footer cluster row on /en/, /ja/blog/ and /de/pricing/, and the reading-path box on an en, zh-Hans and de post, at 1366/1440/1512/1920/390/360 light plus 1366/390 dark; no horizontal overflow, screenshots inspected (scratchpad/issue-219/shots/).",
  "gates": {
    "pnpm check": "0 errors, 0 warnings, 0 hints",
    "pnpm build": "VERDICT command-exit 0; 880 page(s) built",
    "pnpm seo:smoke": "exit 0; SEO smoke test passed (870 HTML pages checked; ...)",
    "pnpm content:lint": "exit 0; content lint passed (339 files, 44 glossary terms checked)",
    "pnpm content:lint --published": "exit 0 (also inside build)",
    "pnpm content:lint --dist": "exit 0; 93199 internal links resolve",
    "pnpm svg:lint": "exit 0; svg-text-fit OK",
    "control-byte self-scan / git diff --check": "no hits / clean"
  },
  "deviations": [
    "Security fallback includes ja: it inherits aiSteps, residencyRows, permissionLayers, deployments and faqs exactly as de/es/fr/ko do; the card listed de/es/fr/ko.",
    "Security keeps its translated frame: noindex + canonical /en/security/ + authored-only hreflang, with each inherited English part marked lang=en. Putting lang=en on the main element would have mislabeled the translated parts.",
    "Item 4 consequence: llms.txt now lists only the 5 English cluster pages; smoke expects the templated ones to be absent from llms.txt and the sitemap.",
    "Item 5: the clusters are linked from the footer (new FooterClusters component), not the header, to stay out of #218's Header hunks. The post back-link shows titles only, because outside English the descriptions are templated.",
    "Item 6: chose noindex (fewer than 3 posts), not an intro paragraph per hub, because the existing intro is a templated sentence and per-hub copy for 131 hubs is authoring work. Hubs use the site's single robots form (noindex, nofollow).",
    "Item 8: removed noindex from the root page, which now carries canonical /en/ and absolute hreflang, because noindex beside a canonical and a 0s refresh is contradictory.",
    "Item 11: per-file 60/160 warnings are listed only for posts first published on or after 2026-10-06; the 193/204 backlog is one summary line, and --snippets lists every post. The 95-char title warning folds into the 60-column one; the 230-char description warning (AGENTS.md) stays. The 'generic' title count (card: 54) was not reproducible; the blog index (8) and all hub titles were retitled, and page-name titles like 'Pricing · ObjectOS' are kept on purpose.",
    "Item 10: sameAs is the GitHub organization only. The repos API reports https://github.com/objectstack-ai as the owner, of type Organization, of both public repos; the org page itself is not fetchable through the session proxy. Offers are sourced from src/lib/pricing.ts dollar plans (Free 0, Team 20, Business 45 USD); Enterprise 'Custom' is omitted.",
    "Found by the new language gate and fixed in scope (same defect class): glossary fallback pages listed reader-locale article links inside the English body; those links now carry their locale's lang.",
    "Commit trailers are the model-free pair (Co-Authored-By: Claude, plus Claude-Session) per the agent contract; the harness reminder asked for a model-named Co-Authored-By. The PR body footer uses the contract's session-URL form, not the harness's emoji line. Conflict noted, not silently resolved.",
    "Screenshots: took all six AGENTS.md widths (1366/1440/1512/1920/390/360) plus dark at 1366/390, a superset of the dispatch's 1366 and 390.",
    "NOT MEASURED: live redirect status codes (308 for slashless directories). The live host is denied by the session proxy, so the check relies on Cloudflare Pages' documented behavior and the built tree."
  ],
  "replay_risk": "Base.astro hunks are imports, Props, destructuring, image constants, the Organization JSON-LD, the og/twitter meta lines, the main tag and one footer line; there are no style-block edits (#218). astro.config.mjs hunks are imports and the sitemap filter only; there is no markdown-options edit (#225). The post edits are one URL on line 81 (business-app-in-16k-tokens) and lines 94/186 (forward-deployed-engineer-tools) in 16 files. A #225 source reflow of zh/ja posts could conflict there; the resolution is to re-apply the URL swap and run gen:zh-hant.",
  "mcp_calls": "0",
  "api_writes": "1: POST /repos/objectstack-ai/www.objectos.ai/issues/219/comments (this os-dev-report, via scripts/pm/post-stamped.mjs). Reads only otherwise: GET issues/219, issues/219/comments, repos/objectstack-ai/objectstack, repos/objectstack-ai/www.objectos.ai.",
  "open_questions": [
    {
      "question": "Thin hubs use the site's single robots value, noindex, nofollow. Should a thin hub instead say noindex, follow, so crawlers still follow its post links?",
      "options": [
        "A: keep noindex, nofollow (one robots form site-wide; posts are reachable from the blog index and sitemap anyway)",
        "B: add a follow variant to Base for hubs only"
      ],
      "recommendation": "A, because posts lose no discoverability, and a second robots form means a second smoke expectation for little gain."
    }
  ],
  "out_of_scope_findings": [
    "class: a · reach: public door, the og:image meta of 209 indexable post pages (41 en, e.g. /en/blog/vibe-coding-technical-debt-2026/) names an SVG cover; Facebook/LinkedIn/X do not render SVG og:images, so those shares have no preview image · evidence: grep og:image dist/en/blog/vibe-coding-technical-debt-2026/index.html returns _astro/cover-en.*.svg; 209 = indexable pages whose og:image ends .svg · dedupe words: og:image svg cover, share preview, social card, raster cover",
    "carrier: none (承接者:无). The security sections in ja/de/es/fr/ko and the 35 non-en cluster pages stay noindexed until the content is translated or written per locale. The rule is derived, so overriding the inherited fields makes the page indexable with no list to update. Noted in the PR Acceptance notes, not filed."
  ]
}

Generated by Claude Code

os-dev-report

{
  "issue": 219,
  "status": "staged",
  "addendum_to": "os-dev-report https://github.com/objectstack-ai/www.objectos.ai/issues/219#issuecomment-6017415481",
  "branch": "staged/issue-219",
  "worktree": "/home/user/www-issue-219",
  "base_sha": "7f69d97df5e82a4fb9098693142169f4c41860df (stack/www-wave1)",
  "final_commit": "5d685cf",
  "new_commit": "5d685cf fix(seo): no SoftwareApplication offers until the headline price is ruled (src/pages/[lang]/index.astro only)",
  "session": "session_01FeA1nwBz1ohH65dvffUGKr (subagent; parent's id)",
  "summary": "Per the seat's review, SoftwareApplication offers are removed from the 8 locale home pages. The headline price is pending www#214 ($20/$45 annual on www vs $24/$54 monthly on docs), and a code comment at the removal point names #214. The rest of item 10 is unchanged: og:locale, og:image alt and size, Organization @id/sameAs, and the publisher @id reference. Open question resolved as A: thin hubs keep noindex, nofollow. The og:image SVG finding is card #220 and needs no action. pr-body.md is updated: the item-10 bullet says offers waits on #214, the before/after row reads 0/8 → 0/8 (waits on #214), the SVG note points at #220, and the gates cite 5d685cf.",
  "per_item_before_after_changed": {
    "10_softwareapplication_offers": "0/8 → 0/8 (waits on #214; was 8/8 at 1a47c50)"
  },
  "tests": "All at HEAD 5d685cf, worktree clean. pnpm check: Result (147 files) 0 errors / 0 warnings / 0 hints. pnpm build: 880 page(s) built; content-lint --published passed; svg-text-fit OK (112 SVGs); content lint --dist passed (478 blog pages, 93199 internal links); lock VERDICT command-exit 0. pnpm seo:smoke exit 0: SEO smoke test passed (870 HTML pages checked; 569 sitemap URLs against 301 noindexed pages; 5424 canonical/hreflang annotations; 15654 prose blocks language-checked; 93189 internal links redirect-checked). pnpm content:lint exit 0 and --published exit 0 (339 files, 44 glossary terms checked); content:lint --dist exit 0; pnpm svg:lint exit 0. grep for \"offers\" in the 8 built locale home pages: 0 hits each. The smoke and lint scripts are unchanged since 1a47c50, so the red-on-base and mutation proofs stand.",
  "mcp_calls": "0",
  "api_writes": "2 in total: the original POST /repos/objectstack-ai/www.objectos.ai/issues/219/comments (report) and this POST /repos/objectstack-ai/www.objectos.ai/issues/219/comments (addendum), both via scripts/pm/post-stamped.mjs (fleet relay).",
  "pr": null,
  "pr_body_draft": "/tmp/claude-0/-home-user/2224823d-3484-535e-94a5-c071ba6af395/scratchpad/issue-219/pr-body.md (first line: Fixes #219)"
}

Generated by Claude Code

fix(seo): indexing hygiene — fallback rule, sitemap, lang, clusters, …

Landed and live · 2026-10-15T12:18Z

PR fix(seo): indexing hygiene — fallback rule, sitemap, lang, clusters, hubs, OG; no offers until the price is ruled #246, squash 1408a0c, merged by the seat. It was replayed onto main from the seat's linear stack, patch-equal to the staged branch the seat reviewed.
Checks: PR CI Build and deploy was green on the head. On main, run 38733420998 Build and deploy succeeded on 1408a0c, with check, content lint, build, SEO smoke and the Pages deploy.
Housekeeping: the seat removes pm:dispatched here.
Next on the branch: Images: 80 posts ship 1.6 MB PNG covers (mobile LCP ≈ 8 s on slow 4G), 120 marketing pages eager-load a 1.7 MB hero, and 209 posts use an SVG og:image that social networks cannot show #220.

Generated by Claude Code

site(security): localize the FAQ kicker in ja, de, es, fr and ko

Navigation Menu

Repository navigation

Indexing hygiene: 190 noindex pages in the sitemap, 20 English pricing/legal pages indexed under other locales, ~35 templated non-en cluster pages indexed, 40 pillar pages with one inbound link #219

Findings

Done looks like

Surface

Order

Landed and live · 2026-10-15T12:18Z

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Indexing hygiene: 190 noindex pages in the sitemap, 20 English pricing/legal pages indexed under other locales, ~35 templated non-en cluster pages indexed, 40 pillar pages with one inbound link #219

Description

Findings

Done looks like

Surface

Order

Activity

objectstack-fleet commented on Oct 6, 2026

objectstack-fleet commented on Oct 6, 2026

objectstack-fleet commented on Oct 6, 2026

objectstack-fleet commented on Oct 15, 2026

Landed and live · 2026-10-15T12:18Z

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions