#ip-blocking
2 posts tagged with ip-blocking.
Blocking AI Crawlers by IP and ASN (for Stealth Scrapers)
Replaced in November 2026: this answer now sits in a longer, current page on the same question. Linked here.
#ai-crawlers #ip-blocking #asn
How to Block AI Scrapers: The Complete Enforcement Guide (2026)
robots.txt won't stop scrapers that ignore it. The enforcement layer, with working config: rate limiting, bot verification, WAF rules, IP and ASN blocking, and tarpits as a last resort.
#ai-crawlers #enforcement #waf