#ip-blocking
2 posts tagged with ip-blocking.
-
Blocking AI Crawlers by IP and ASN (for Stealth Scrapers)
When scrapers spoof user-agents, block by IP and ASN instead. How to use network-level blocking and rate limiting to stop stealth AI crawlers that ignore robots.txt and fake their identity.
-
How to Block AI Scrapers: The Complete Enforcement Guide (2026)
robots.txt won't stop scrapers that ignore it. This is the enforcement layer: WAF rules, bot verification by IP range, IP/ASN blocking, rate limiting, and tarpits — how to actually keep non-compliant AI crawlers out.