AI Scraping on Creators' Work Reaches a New Level

Patreon's New AI Blocking Measures

Patreon, a membership platform for creators, has introduced strengthened measures to combat AI scraping. The company is working with Cloudflare to directly block access to AI bots designed to train their AI models on creators' work without permission. This move comes after Patreon realized that AI scraping has become more sophisticated since it first put measures in place to deter AI crawlers in 2023.

• AI scraping has become a significant concern for online publishers and content creators, as it can ingest their work for the purpose of making AI models smarter.
• To combat this, Cloudflare now offers tools that allow website publishers to restrict AI bots, including a marketplace that lets websites charge AI bots for scraping, dubbed Pay Per Crawl.
• Earlier this month, Cloudflare changed its policies so that "mixed-use" crawlers, meaning those that both index and train on a website's content, are blocked by default on any pages that host ads.

Patreon's Partnership with Cloudflare

Patreon is extending its existing work with Cloudflare to use the company's AI Crawl Control technology to update its AI policies and enforcement tools. The company is now actively blocking AI training bots, rather than simply asking them not to scrape content using the robots.txt files.

"Consent shouldn't depend on whether a scraper chooses to behave," a Patreon blog post explains, referencing the stricter measures.

Effectiveness of the New Measures

When testing the features, individual AI training crawlers' weekly attempts to access Patreon went from "thousands of attempts to zero," the post noted. This indicates that the AI scrapers were ignoring Patreon's robots.txt file and scraping the site anyway, despite its requests.

• However, Patreon will allow bots that index pages and organize information that can be used to send users back to Patreon.
• The company's product chief Drew Rowny remarked, "As AI agents become increasingly powerful and popular, creators deserve a meaningful say in how their work is used by AI companies."
• Patreon has a different vision: creators should be able to grow their audience and control how their work is used.