Reddit blocks AI bots from crawling its website

  • 📰 verge
  • ⏱ Reading Time:
  • 23 sec. here
  • 2 min. at publisher
  • 📊 Quality Score:
  • News: 13%
  • Publisher: 67%

Technology Technology Headlines News

Technology Technology Latest News,Technology Technology Headlines

Reddit is updating its robots.txt file to block all data scrapers it doesn’t have commercial agreements with, according to a company executive.

In the coming weeks, Reddit will start blocking most automated bots from accessing its public data. You’ll need to make a licensing deal, like Google and OpenAI have done, to use Reddit content for model training and other commercial purposes. While this has technically been Reddit’s policy already, the company is now enforcing it by updating its robots.txt file, a core part of the web that dictates how web crawlers are allowed to access a site.

Lee has seen this story play out before from the other side of the fence; long before he worked at Reddit, he was senior legal counsel at Google in the early 2000s. Back then, it was Google that was speedrunning the legal system to build up Search and YouTube. Now, the internet is being reshaped again by the rise of generative AI. For companies like Reddit, the risk of AI subsuming everything is too value destructive to not fight against.

 

Thank you for your comment. Your comment will be published after being reviewed.
Please try again later.
We have summarized this news so that you can read it quickly. If you are interested in the news, you can read the full text here. Read more:

 /  🏆 94. in TECHNOLOGY

Technology Technology Latest News, Technology Technology Headlines

Similar News:You can also read news stories similar to this one that we have collected from other news sources.

Reddit puts AI scrapers on noticeKarissa is a senior reporter at Engadget, covering Meta, Twitter, TikTok, Snap and all things social media. Her interests include tech policy, internet culture, and all the ways our online activities shape our IRL selves.
Source: engadget - 🏆 276. / 63 Read more »