Photo by 112 Uttar Pradesh on Pexels

A new initiative called Really Simple Licensing (RSL) is gaining traction, offering a standardized way for web publishers to license their content for AI training purposes. Supported by platforms like Reddit, Yahoo, Medium, and Quora, RSL enables publishers to define the terms under which AI developers can utilize their web data.

Building upon the established robots.txt protocol, RSL adds layers of licensing and royalty options. Websites can now specify subscription fees, per-crawl charges, or even per-inference fees, allowing them to receive compensation when AI models utilize their content. The RSL Collective, spearheaded by Eckart Walther and Doug Leeds, aims to streamline the licensing process for website owners, supporting diverse models including free options.

While RSL doesn’t inherently block bots, the Collective is partnering with Fastly to gate access based on licensing agreements. The success of RSL hinges on adoption by leading AI companies, especially given past accusations of ignoring robots.txt. Currently, large media organizations like Vox Media, News Corp, and The New York Times have already established individual licensing deals with entities like OpenAI and Amazon. RSL aims to democratize access to compensation, providing a simpler path for all website owners. The effort comes amidst legal uncertainties surrounding unauthorized web scraping for AI training, with RSL seeking to provide clear licensing terms upfront.