Skip to content

fix: add Amazonbot and MistralAI-Training to the AI training opt-out - #352

Merged
harlan-zw merged 1 commit into
nuxt-modules:mainfrom
daniel63194:fix/ai-training-gaps
Oct 12, 2026
Merged

harlan-zw merged 1 commit into
nuxt-modules:mainfrom
daniel63194:fix/ai-training-gaps

Conversation

@daniel63194

Copy link
Copy Markdown
Contributor

🔗 Linked issue

Follow-up to #347 and #349.

❓ Type of change

  • 📖 Documentation (updates to the documentation or readme)
  • 🐞 Bug fix (a non-breaking change that fixes an issue)
  • 👌 Enhancement (improving an existing functionality)
  • ✨ New feature (a non-breaking change that adds functionality)
  • 🧹 Chore (updates to the build process or auxiliary tools and libraries)
  • ⚠️ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description

Thanks for shipping blockAiBots: 'training' so quickly! Comparing AiTrainingBots with the operators' docs, two documented training crawlers are missing:

  • MistralAI-Training: Mistral's crawler for building training datasets (docs). It was missing from AiBots too, so neither true nor 'training' blocked it. MistralAI-Index and MistralAI-User are already listed and stay out of the training list.
  • Amazonbot: Amazon says Amazonbot "may be used to train Amazon AI models" (docs). Amazon's search and answer agents, Amzn-SearchBot and Amzn-User, are separate and, per the same page, not used for generative AI training, so they stay unblocked.

Changes:

  • src/const.ts: adds both tokens to AiTrainingBots with their operator sources, and MistralAI-Training to AiBots.
  • test/unit/const.test.ts: AiTrainingBots must be a subset of AiBots, so true always blocks everything 'training' blocks. Also checks the two tokens and that the Amazon and Mistral search/user agents stay out.
  • test/e2e/ai-training.test.ts and the blockAiBots docs list the two new tokens.

@harlan-zw

harlan-zw commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

🤖 MERGED

Harlan Agent Kit posted this automated review. It is not Harlan's personal review or approval. AI open source policy. Last updated: 2026-10-12 17:24 UTC.

GitHub merged this pull request.

No material findings were recorded.

411fd30e-cbff-499c-a07c-29291b76e5e3

@pkg-pr-new

pkg-pr-new Bot commented Oct 12, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@nuxtjs/robots@352

commit: 0d8bfcc

@github-actions

Copy link
Copy Markdown

Nuxt logo Nuxt Module Size Analyzer

No notable runtime size changes.

Module Client gzip Server raw
@nuxtjs/robots 608 B 173 kB
Package files (6)

Rows overlap and include unused code. These totals do not measure deployed output or npm downloads.
Excludes dependency files, types, source maps, and other non-code files.

Package files Summed gzip Raw Δ gzip
@nuxtjs/robots · export . 7.1 kB 27 kB —
@nuxtjs/robots · export ./content 292 B 503 B —
@nuxtjs/robots · export ./util 467 B 784 B —
@nuxtjs/robots · built code files 44 kB 142 kB 🔴 +156 B (+0.4%)
@nuxtjs/robots · app source files 3.3 kB 7.2 kB —
@nuxtjs/robots · server source files 9.8 kB 24 kB —
Dependencies (8)

Declared dependencies. Their installed size does not show runtime cost.

Package Dependency Requested
@nuxtjs/robots @fingerprintjs/botd catalog:
@nuxtjs/robots @nuxt/kit catalog:
@nuxtjs/robots @vueuse/core catalog:
@nuxtjs/robots consola catalog:
@nuxtjs/robots defu catalog:
@nuxtjs/robots nuxt-site-config catalog:
@nuxtjs/robots nuxtseo-shared catalog:
@nuxtjs/robots ufo catalog:
How this is measured
  • Compare the same app with and without the module.
  • Client: all emitted JS/CSS, summed per-file gzip. This includes lazy chunks.
  • Server: deployed files, including external dependencies. Source maps are excluded.
  • Default module settings, Node server preset. Memory and request speed are outside this report.

Base: main @ fab5a9e · 2026-10-12 · differences below 16 B are ignored

@harlan-zw

harlan-zw commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Harlan Agent Kit posted this automated status. AI open source policy.

Published @nuxtjs/robots@7.0.1.

@harlan-zw harlan-zw added harlan-agent-review Approve automated work for the current issue state or pull request head commit. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-ready The automated Review passed every gate on this head commit. and removed harlan-agent-review Approve automated work for the current issue state or pull request head commit. harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. labels Oct 12, 2026
@harlan-zw
harlan-zw merged commit 0c0e458 into nuxt-modules:main Oct 12, 2026
8 checks passed
@harlan-zw

Copy link
Copy Markdown
Contributor

Good catch, thanks :)

@harlan-zw harlan-zw mentioned this pull request Oct 12, 2026
6 tasks
@harlan-zw harlan-zw removed the harlan-agent-ready The automated Review passed every gate on this head commit. label Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants