From 74ca62cb573d4dd55b3d20360d37cf756d268103 Mon Sep 17 00:00:00 2001 From: Harlan Wilton Date: Tue, 13 Oct 2026 00:33:36 +1100 Subject: [PATCH 1/3] docs: add the v7 release notes and Nuxt 4.6 install floor Robots 7 ships in Nuxt SEO v6, and the live docs had no v7 page. The install page now states the Nuxt and Node floor. Examples now use the public aliases and RequestEvent fields. --- README.md | 5 +- .../1.getting-started/1.installation.md | 14 ++ .../1.getting-started/3.troubleshooting.md | 1 + docs/content/2.guides/1.robots-txt.md | 18 +- docs/content/2.guides/2.how-it-works.md | 2 +- docs/content/2.guides/5.disable-indexing.md | 6 +- docs/content/2.guides/8.bot-detection.md | 8 +- docs/content/4.api/3.use-bot-detection.md | 10 +- .../5.nitro-api/2.get-bot-detection.md | 6 +- docs/content/5.nitro-api/3.nitro-hooks.md | 4 +- docs/content/6.releases/2.v7.md | 196 ++++++++++++++++++ 11 files changed, 244 insertions(+), 26 deletions(-) create mode 100644 docs/content/6.releases/2.v7.md diff --git a/README.md b/README.md index c8cbf9a1..aa359ee1 100644 --- a/README.md +++ b/README.md @@ -40,9 +40,12 @@ need these features. Install `@nuxtjs/robots` dependency to your project: ```bash -npx nuxi@latest module add robots +npx nuxt module add robots ``` +> [!TIP] +> Using an AI agent? Get the @nuxtjs/robots Skill on [skilld.dev/gh/nuxt-modules/robots](https://skilld.dev/gh/nuxt-modules/robots). + 💡 Configured your robots.txt? Verify what crawlers and AI bots can access with the free [Robots.txt Validator](https://nuxtseo.com/tools/robots-txt-validator), or monitor crawl & indexing status with [Nuxt SEO Pro](https://nuxtseo.com/pro). ## Documentation diff --git a/docs/content/1.getting-started/1.installation.md b/docs/content/1.getting-started/1.installation.md index 56b990de..10778d16 100644 --- a/docs/content/1.getting-started/1.installation.md +++ b/docs/content/1.getting-started/1.installation.md @@ -20,6 +20,16 @@ To get started with Nuxt Robots, you need to install the dependency and add it t :ModuleInstall{name="robots" included-in="@nuxtjs/seo"} +Use Nuxt 4.6 or Nuxt 5. +Use Node 22.22.3+, Node 24.15.0+, or Node 26+. + +Import app functions from `#robots/app`. Import server functions from `#robots/server`. +If you use Nuxt Content, use version 3. + +::tip +Using an AI agent? Get the @nuxtjs/robots Skill on [skilld.dev/gh/nuxt-modules/robots](https://skilld.dev/gh/nuxt-modules/robots). +:: + ## Verifying Installation To ensure the module is behaving as expected, you should first check [`/robots.txt`](http://localhost:3000/robots.txt) is being generated. @@ -31,6 +41,10 @@ However, we want to see what a production environment would look like. For this, it's recommended to use the Nuxt DevTools Robots tab to see the current configuration and how it's being applied. +::note +The DevTools panel needs the `nuxtseo-devtools-host` package. Install it as a dev dependency with `pnpm add -D nuxtseo-devtools-host`, then restart the dev server. You do not add it to `modules`. +:: + The DevTools will show you that in production we're just serving a minimal robots.txt file. ```robots-txt [robots.txt] diff --git a/docs/content/1.getting-started/3.troubleshooting.md b/docs/content/1.getting-started/3.troubleshooting.md index 80f8bb50..a3cdea76 100644 --- a/docs/content/1.getting-started/3.troubleshooting.md +++ b/docs/content/1.getting-started/3.troubleshooting.md @@ -15,6 +15,7 @@ relatedPages: ## Debugging **Nuxt DevTools.** The best tool for debugging is the Nuxt DevTools integration with Nuxt Robots. This will show you the current robot rules and your robots.txt file. +The panel needs the `nuxtseo-devtools-host` package as a dev dependency. Install it with `pnpm add -D nuxtseo-devtools-host`. **Debug config.** You can enable the [debug](/docs/robots/api/config#debug) option which will give you more granular output. This is enabled by default in development mode. diff --git a/docs/content/2.guides/1.robots-txt.md b/docs/content/2.guides/1.robots-txt.md index b1bddce0..1bf30d13 100644 --- a/docs/content/2.guides/1.robots-txt.md +++ b/docs/content/2.guides/1.robots-txt.md @@ -31,21 +31,23 @@ Additionally, the following paths are supported by default: ```bash [Example File Structure] # root directory robots.txt -# asset folders -assets/ -├── robots.txt -# pages folder -pages/ -├── robots.txt -├── _dir/ -│ └── robots.txt # public folder public/ ├── _robots.txt ├── _dir/ │ └── robots.txt +# source directory (app/ on Nuxt 4) +app/ +├── assets/ +│ └── robots.txt +└── pages/ + ├── robots.txt + └── _dir/ + └── robots.txt ``` +The module reads the `assets/` and `pages/` paths from your `srcDir`. On Nuxt 4, the default `srcDir` is `app/`. + **Custom paths.** If you find this too restrictive, you can use the `mergeWithRobotsTxtPath` config to load your `robots.txt` file from any path. diff --git a/docs/content/2.guides/2.how-it-works.md b/docs/content/2.guides/2.how-it-works.md index 4dd4e9ab..8be4b7e9 100644 --- a/docs/content/2.guides/2.how-it-works.md +++ b/docs/content/2.guides/2.how-it-works.md @@ -27,7 +27,7 @@ public ``` This file is generated differently depending on the environment: -- When deploying using `nuxi generate` or the `nitro.prerender.routes` rule, this is a static file. +- When deploying using `nuxt generate` or the `nitro.prerender.routes` rule, this is a static file. - Otherwise, it's handled by the server and generated at runtime when requested. When indexing is disabled a `robots.txt` will be generated with the following content: diff --git a/docs/content/2.guides/5.disable-indexing.md b/docs/content/2.guides/5.disable-indexing.md index bf8d9c15..e8695924 100644 --- a/docs/content/2.guides/5.disable-indexing.md +++ b/docs/content/2.guides/5.disable-indexing.md @@ -96,10 +96,12 @@ to whatever makes sense for your project. If you deploy the same app to multiple domains (e.g. `mydomain.com` and `test.temp-domain.com`), you can disable indexing for specific hosts at runtime using a Nitro plugin. ```ts [server/plugins/robots-domain.ts] +import { getNitroOrigin } from '#site-config/server' + export default defineNitroPlugin((nitroApp) => { nitroApp.hooks.hook('site-config:init', ({ event, siteConfig }) => { - const host = event.headers.get('host') - if (host?.includes('staging') || host?.includes('test')) { + const host = new URL(getNitroOrigin(event)).hostname + if (host.includes('staging') || host.includes('test')) { siteConfig.push({ indexable: false, _context: 'host-indexing' }) } }) diff --git a/docs/content/2.guides/8.bot-detection.md b/docs/content/2.guides/8.bot-detection.md index 5f063425..763421e1 100644 --- a/docs/content/2.guides/8.bot-detection.md +++ b/docs/content/2.guides/8.bot-detection.md @@ -95,7 +95,7 @@ Since server-side detection only uses HTTP headers, detection can only work for You can detect bots inside a Nitro route, middleware, or API handler. ```ts -import { getBotDetection } from '#robots/server/composables/getBotDetection' +import { getBotDetection } from '#robots/server' export default defineEventHandler((e) => { const detection = getBotDetection(e) @@ -116,7 +116,7 @@ When using bot detection in Nuxt, it will use the `User-Agent` header by default ```vue @@ -103,15 +105,18 @@ export default defineNuxtConfig({ }) ``` -Server examples use Nitro auto-imports. On Nuxt 4.6, importing them from `#imports` in a server file fails `nuxt typecheck`. +Import server functions from `#robots/server` and app composables from `#robots/app`. Auto-imports also work. +Do not import server functions from `#imports`. On Nuxt 4.6, that fails `nuxt typecheck`. Turn off indexing for one host of a multi-domain deployment. This is per request: ```ts // server/plugins/robots-host.ts +import { getNitroOrigin } from '#site-config/server' + export default defineNitroPlugin((nitroApp) => { nitroApp.hooks.hook('site-config:init', ({ event, siteConfig }) => { - if (getRequestHost(event).startsWith('staging.')) + if (new URL(getNitroOrigin(event)).hostname.startsWith('staging.')) siteConfig.push({ _context: 'staging-host', indexable: false }) }) }) @@ -123,17 +128,20 @@ Detect bots from the `User-Agent` header: ```ts // server/middleware/block-bots.ts +import { createError, defineEventHandler } from 'nuxt/server' +import { getBotDetection } from '#robots/server' + export default defineEventHandler((event) => { const bot = getBotDetection(event) // { isBot, botName, botCategory, trusted } // Search engines and AI crawlers count as trusted. To block AI crawlers, test bot.botCategory === 'ai'. - if (event.path.startsWith('/api/') && bot.isBot && !bot.trusted) - throw createError({ statusCode: 403 }) + if (event.url.pathname.startsWith('/api/') && bot.isBot && !bot.trusted) + throw createError({ status: 403 }) }) ``` In the app, `useBotDetection()` returns the same fields as computed refs. `useBotDetection({ fingerprint: true })` also runs BotD in the browser. Its result arrives later, so watch `isBot`. -Outside Nuxt, `getBotDetection(headers)` from `@nuxtjs/robots/util` reads only a lowercase `user-agent` key. Pass `Object.fromEntries(request.headers)`. A `Headers` instance returns `isBot: false`. +Outside Nuxt, `getBotDetection(headers)` from `@nuxtjs/robots/util` accepts a `Headers` instance or a plain header object. ## Integrations @@ -142,25 +150,29 @@ Nuxt Content v3 frontmatter and Nuxt i18n path expansion: [references/integratio ## Traps - A path can be crawlable and noindex, or blocked in robots.txt. It cannot be both. Google never reads `noindex` on a disallowed URL and can still index it from links. -- `useRobotsRule(true)` sends `robotsEnabledValue` even when the site is not indexable. Pass `true` only to override that. +- `useRobotsRule(true)` keeps `noindex` when the site is not indexable. To force a rule, pass the string, such as `useRobotsRule('index, follow')`. +- `useRobotsRule(null)` and `useRobotsRule(undefined)` keep the existing rule. - `useRobotsRule()` sets the rule only during SSR. In the browser it only reads the rendered value. - `noai` or `noimageai` makes a path non-indexable to the module, even with `index: true`. `getPathRobotConfig` returns `indexable: false`, and `disallowNonIndexableRoutes` disallows the path. - An `app.baseURL` other than `/` turns off robots.txt and logs an error. Crawlers only read `/robots.txt` at the host root. - Nuxt Content frontmatter `robots` sets the meta tag but not the header, and it needs `zod` 4 in your dependencies. See [references/integrations.md](references/integrations.md). -- The key is `blockAiBots`. Some docs prose spells it `blockAIBots`. +- The key is `blockAiBots`, not `blockAIBots`. - Use `blockAiBots: 'training'` to leave dedicated AI search and user fetch agents unblocked. Existing rules still apply. - Training mode includes Google-Extended, which also controls grounding in Gemini Apps and Vertex AI. ## Version limits -These hold for 6.2.4: - -- `definePageMeta({ robots })` has no effect on Nuxt 4.6. Use a route rule, `defineRouteRules()`, or `useRobotsRule()`. -- A build that calls `useBotDetection()` fails with `Rolldown failed to resolve import "@vueuse/core"` unless the app installs `@vueuse/core`. -- `robots.disableNuxtContentIntegration` has no effect. -- The docs' `robots:config` Nitro hook that edits `ctx.groups` per host leaks. The result is shared, so other hosts get `noindex` until the next `/robots.txt` request. Use the `site-config:init` plugin above. -- `groups[].comment` must be an array. A string prints one `#` line per character. -- `disallowNonIndexableRoutes: true` (deprecated) also writes `Disallow: /_nuxt` and `Disallow: /_nuxt/*` into robots.txt. +Version 7 breaks these 6.x patterns: + +- Nuxt 3 and Node 20 are unsupported. Use Nuxt `^4.6.0 || ^5.0.0` and Node `^22.22.3 || ^24.15.0 || >=26.0.0`. +- `#internal/nuxt-robots` is removed. Import from `#robots/server` or `#robots/app`. +- Hook events are `RequestEvent` from `nuxt/server`. Read `ctx.event.req.headers.get('host')`, not `ctx.event.headers` or `event.node.req`. +- A `robots:config` edit made with `ctx.event` changes only that `/robots.txt` response. For per host indexing, use the `site-config:init` plugin above. +- A route rule beats `definePageMeta({ robots })` on the same path. +- `assets/robots.txt` and `pages/robots.txt` load from `srcDir`. On Nuxt 4, move them into `app/`. +- Nuxt Content v2 is unsupported, and Nuxt fails at startup with a version error. Upgrade to Nuxt Content v3. +- `defineRobotsSchema()` needs `zod` `^4.6.5`. +- The DevTools panel needs `nuxtseo-devtools-host` as a dev dependency: `pnpm add -D nuxtseo-devtools-host`. ## Config diff --git a/skills/nuxtjs-robots/references/integrations.md b/skills/nuxtjs-robots/references/integrations.md index b0c6a01c..b0b30c5f 100644 --- a/skills/nuxtjs-robots/references/integrations.md +++ b/skills/nuxtjs-robots/references/integrations.md @@ -33,7 +33,7 @@ const route = useRoute() const { data: page } = await useAsyncData(`page-${route.path}`, () => queryCollection('content').path(route.path).first()) // The meta tag comes from page.seo. Only useRobotsRule() also sets the header. useSeoMeta(page.value?.seo || {}) -// Content returns null for an unset key. null sends X-Robots-Tag: null and drops the meta tag. +// Content returns null for an unset key. Robots 7 ignores null and undefined, so the default rule stays. useRobotsRule(page.value?.robots ?? undefined) From 5be66dfc1d87cf16e95fe8b83efacaab87797511 Mon Sep 17 00:00:00 2001 From: Harlan Wilton Date: Tue, 13 Oct 2026 01:09:20 +1100 Subject: [PATCH 3/3] docs: tighten v7 release wording A pass for upgrading readers: exact Node and Zod ranges in the upgrade steps, the default srcDir stated as a default, and the null rule change separated from the existing undefined behaviour. --- docs/content/1.getting-started/3.troubleshooting.md | 2 +- docs/content/6.releases/2.v7.md | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/content/1.getting-started/3.troubleshooting.md b/docs/content/1.getting-started/3.troubleshooting.md index a3cdea76..99473f25 100644 --- a/docs/content/1.getting-started/3.troubleshooting.md +++ b/docs/content/1.getting-started/3.troubleshooting.md @@ -15,7 +15,7 @@ relatedPages: ## Debugging **Nuxt DevTools.** The best tool for debugging is the Nuxt DevTools integration with Nuxt Robots. This will show you the current robot rules and your robots.txt file. -The panel needs the `nuxtseo-devtools-host` package as a dev dependency. Install it with `pnpm add -D nuxtseo-devtools-host`. +The panel needs the `nuxtseo-devtools-host` package as a dev dependency. Install it with `pnpm add -D nuxtseo-devtools-host`, then restart the dev server. **Debug config.** You can enable the [debug](/docs/robots/api/config#debug) option which will give you more granular output. This is enabled by default in development mode. diff --git a/docs/content/6.releases/2.v7.md b/docs/content/6.releases/2.v7.md index a8f3fa19..7f1a51f9 100644 --- a/docs/content/6.releases/2.v7.md +++ b/docs/content/6.releases/2.v7.md @@ -103,7 +103,7 @@ See [Handling Multiple Domains](/docs/robots/guides/disable-indexing#handling-mu `useRobotsRule(true)` now sends `robotsDisabledValue` when Site Config disables indexing. ([#340](https://github.com/nuxt-modules/robots/pull/340)) In v6, it sent `robotsEnabledValue`, so a staging page could become indexable. -`null` and `undefined` now keep the existing rule. +`null` now keeps the existing rule, as `undefined` already did. ### Route Rules Take Precedence Over Page Metadata @@ -113,12 +113,12 @@ When a route rule and `definePageMeta({ robots })` both match a path, the route The module reads `assets/robots.txt`, `pages/robots.txt`, and `pages/_dir/robots.txt` from `srcDir`. ([#340](https://github.com/nuxt-modules/robots/pull/340)) -On Nuxt 4, `srcDir` is `app/`. Move these files to `app/assets/` or `app/pages/`. +On Nuxt 4, the default `srcDir` is `app/`. If you keep that default, move these files to `app/assets/` or `app/pages/`. The `public/` and root directory paths do not change. ### Zod 4 for `defineRobotsSchema()` -The optional `zod` peer dependency is now `^4.6.5`. Install Zod 4 to use `defineRobotsSchema()`. ([#333](https://github.com/nuxt-modules/robots/pull/333)) +The optional `zod` peer dependency is now `^4.6.5`. If you use `defineRobotsSchema()`, install Zod 4.6.5 or a later 4.x release. ([#333](https://github.com/nuxt-modules/robots/pull/333)) ### DevTools Needs `nuxtseo-devtools-host` @@ -186,7 +186,7 @@ Coding agents read it from `node_modules/@nuxtjs/robots/skills/`. ## Upgrading -1. Upgrade to Nuxt 4.6 or Nuxt 5, and Node 22.22.3 or later. +1. Upgrade to Nuxt 4.6 or Nuxt 5. Use Node 22.22.3+, 24.15.0+, or 26+. 2. If you use `@nuxtjs/seo`, follow the [Nuxt SEO v5 to v6 migration guide](/docs/nuxt-seo/migration-guide/v5-to-v6). Otherwise, install `@nuxtjs/robots@^7`. 3. Replace `#internal/nuxt-robots` and deep `#robots/...` imports with `#robots/server` or `#robots/app`. 4. Replace `event.headers` and `event.node.req` in robots hooks with `event.req.headers`.