Google's AI Crawlers: Google-Extended, Google-CloudVertexBot, and Gemini
Google's AI crawling is confusing because Google-Extended isn't a crawler — it's an opt-out token. Here's how Google-Extended, Google-CloudVertexBot, and Googlebot relate to Gemini training and AI features.