Meta and Amazon AI Crawlers: Meta-ExternalAgent, Meta-ExternalFetcher, Amazonbot

By Andrej Ruckij · June 17, 2026

TL;DR: Meta runs Meta-ExternalAgent (training, for Meta AI / Llama) and Meta-ExternalFetcher (user-fetch, real-time when someone asks Meta AI). Amazon runs Amazonbot — and an easy-to-confuse sibling, Amzn-SearchBot, described as search-only. Block the training crawler if you wish; the older FacebookBot is a separate, pre-AI crawler.

A cluster under the AI crawler directory. Meta and Amazon are lower-profile than OpenAI/Anthropic/Google but worth getting right — especially the Amazon naming trap.

Meta’s AI crawlers

User-agent	Job	Recommendation
`Meta-ExternalAgent`	Train Meta AI / Llama (launched Jul 2024)	Block to opt out of training
`Meta-ExternalFetcher`	Fetch in real time when a user asks Meta AI	Allow (user-fetch / visitor intent)
`FacebookBot`	Older, separate Meta crawler (not the AI-training agent)	Decide separately

Meta-ExternalAgent (UA meta-externalagent/1.1) is Meta’s training crawler — the equivalent of GPTBot or ClaudeBot. Meta-ExternalFetcher is the user-fetch counterpart, firing when someone asks Meta AI about a page. Don’t conflate either with the older FacebookBot, which predates this AI push and does a different job.

User-agent: Meta-ExternalAgent
Disallow: /

Amazon’s crawlers — mind the naming

Amazon is where it’s easy to get it wrong:

Amazonbot — Amazon’s general crawler; it may feed AI features. Respects robots.txt.
Amzn-SearchBot — a sibling described as search-only (powering Alexa/Rufus-style answers rather than training).

Because the two sit close together and Amazon’s documentation is less explicit than OpenAI’s, verify which one you’re seeing before treating them identically. If your goal is to allow search/answer features but limit training-style use, the distinction matters — don’t blanket-block on the Amazon substring.

Recommended setup

Meta: block Meta-ExternalAgent to opt out of Llama/Meta AI training; leave Meta-ExternalFetcher (a visitor) alone.
Amazon: decide on Amazonbot based on your training stance; treat Amzn-SearchBot as a retrieval/answer bot (closer to “allow”). Confirm the exact token rather than assuming.

As always, these are robots.txt-respecting bots; non-compliant scrapers need a firewall (robots-txt-vs-waf-ai-bots), and you should verify any bot by published IP range, not name.

Key takeaways

Meta: Meta-ExternalAgent (training — block to opt out) vs Meta-ExternalFetcher (user-fetch — allow); FacebookBot is separate and older.
Amazon: Amazonbot (general, may feed AI) vs Amzn-SearchBot (search-only) — verify which before treating them the same.
Lower-profile than the big three, but the Amazon naming trap is a real gotcha.

ai-crawler-user-agents-directory — the full cross-vendor bot table
openai-crawlers · anthropic-crawlers — the cleaner-separated equivalents
which-ai-bots-to-block — the overall allow/block policy
glossary/ai-crawler — the training/retrieval/user-fetch taxonomy

Sources

seo/ai-crawler-access — internal synthesis (Meta/Amazon bots + the Amazonbot vs Amzn-SearchBot distinction)