AI Crawler Access Checklist: 8 Steps for Site Owners
Replaced in November 2026: this answer now sits in a longer, current page on the same question. Linked here.
By Andrej Ruckij · · 1 min read
AI Crawler Access Checklist: 8 Steps for Site Owners
This article has been replaced. The setup steps from this checklist, including the command that shows whether bots actually reach your site and the quarterly re-check, are now part of our decision page.
Read the current version: Should you block or allow AI crawlers? A decision rule and the robots.txt to ship