Loading...
Loading...
Browse all stories on DeepNewz
VisitTech companies' reactions to Reddit's robots.txt update by end of 2024?
Google adjusts its crawlers • 33%
OpenAI stops using Reddit data • 33%
Both companies issue a joint response • 34%
Official statements from Google, OpenAI, and other major tech companies
Reddit ($RDDT) Updates Robots.txt to Safeguard Against AI Crawlers
Jun 25, 2024, 07:14 PM
Reddit has announced upcoming changes to its platform aimed at safeguarding against AI crawlers. The company plans to update its robots.txt file to clearly specify that companies using automated agents to access Reddit must adhere to its terms. This move is part of Reddit's broader strategy to block and rate-limit unknown bots and crawlers. Additionally, Reddit will continue to allow good faith actors, such as researchers and organizations like the Internet Archive, to access its content for non-commercial use. These measures come as companies like Google and OpenAI increasingly license Reddit data to train large language models, which has led to concerns about the quality and authenticity of the content being used. The announcement was highlighted by TechCrunch and Reddit's stock symbol is $RDDT.
View original story
Yes • 50%
No • 50%
Twitter restricts search engines • 25%
Facebook restricts search engines • 25%
LinkedIn restricts search engines • 25%
No other major websites follow suit • 25%
Mojeek sees a decline in traffic • 25%
Bing sees a decline in traffic • 25%
DuckDuckGo sees a decline in traffic • 25%
No significant impact on any search engine • 25%
Microsoft • 25%
Perplexity • 25%
Anthropic • 25%
Other • 25%
Implement stricter content usage policies • 25%
Negotiate licensing agreements • 25%
No significant changes • 25%
Challenge NYT legally • 25%
Antitrust lawsuit • 25%
Regulatory fines • 25%
Both • 25%
Neither • 25%
Yes • 50%
No • 50%
No comment • 25%
Express support for Internet Archive • 25%
Criticize Internet Archive's security • 25%
Announce partnerships for better security • 25%
Support Google's stance • 25%
Criticize Google's stance • 25%
No official response • 25%
Mixed responses • 25%
Traffic decreases • 33%
Traffic increases • 33%
Traffic remains stable • 34%