Cloudflare launched a new tool on Tuesday that allows publishers to impose charges on AI bots for scraping their websites. This could drastically change the way the internet functions in the AI age, as models have conventionally scraped content freely and without permission, often irritating content owners. As a key infrastructure provider, Cloudflare will enable significant segments of the internet to utilize this tool.

The “pay per crawl” capability was featured in Cloudflare’s Tuesday announcement, establishing the company as the first internet infrastructure provider to “prevent AI crawlers from accessing content without consent or compensation, by default.” Currently in beta, the tool permits site owners to levy a charge each time an AI bot seeks to “crawl” their website for information.

Cloudflare’s initiative is designed to support content creators whose works are utilized for AI training. (Full disclosure: Ziff Davis, which owns Mashable, was among the publishers advocating for a permission-based methodology regarding AI bots in Cloudflare’s press release.)

Cloudflare communicated in its press release:

“For decades, the Internet has functioned on a straightforward exchange: search engines index content and guide users back to the original websites, generating traffic and ad revenue for sites of all sizes. This cycle rewards creators of high-quality content with income and an audience, while assisting users in discovering new and pertinent information. That model is now compromised. AI crawlers harvest content such as text, articles, and images to produce answers, without directing visitors to the original source – depriving content creators of income, and the gratification of knowing that their content is being viewed.”

The tool may be advantageous for creators of original online content.