How AI Crawlers Work: From Request to Model, Answer, or Visit
How AI crawlers fetch and use your content: the request, user-agent identification, robots.txt check, and the three destinations — model training, a citation index, or a live user's screen.