Perplexity's Crawlers: PerplexityBot, Perplexity-User, and the Stealth-Crawling Controversy
PerplexityBot indexes for citations; Perplexity-User fetches pages users ask about (and ignores robots.txt by design). Plus the August 2025 Cloudflare report that Perplexity crawled sites that blocked it.