Cloudflare Introduces AI Labyrinth to Trap Web Scraping Bots in Endless AI-Generated Loops
Cloudflare’s AI Labyrinth traps web scrapers in AI-generated decoy pages to slow them down and identify bad actors.
Matilda
Cloudflare Introduces AI Labyrinth to Trap Web Scraping Bots in Endless AI-Generated Loops
Cloudflare, one of the biggest network internet infrastructure companies in the world, has announced AI Labyrinth, a new tool to fight web-crawling bots that scrape sites for AI training data without permission. The company says in a blog post that when it detects “inappropriate bot behavior,” the free, opt-in tool lures crawlers down a path of links to AI-generated decoy pages that “slow down, confuse, and waste the resources” of those acting in bad faith. Image: Google Websites have long used the honor system approach of robots.txt, a text file that gives or denies permission to scrapers, but which AI companies, even well-known ones like Anthropic and Perplexity AI, have been accused of ignoring. Cloudflare writes that it sees over 50 billion web crawler requests per day, and although it has tools for spotting and blocking the malicious ones, this often prompts attackers to switch tactics in “a never-ending arms race.” Cloudflare says rather than block bots, AI Labyrinth fights back by…
