Reddit Blocks All Search Engines Except Google in $60 Million AI Deal: 404 Media

Jul 24, 2024, 01:22 PM

In the last year, the number of websites specifically restricting OpenAI and other AI scraper bots has significantly increased. Reddit has blocked all search engines from crawling its site, except Google, which is paying Reddit $60 million a year to scrape it for training data. As a result, Google is now the only search engine that can crawl Reddit. This move has been confirmed by Colin Hayhurst, CEO of the search engine Mojeek, who stated, 'They’re [Reddit] killing everything for search but Google.' This development, reported by 404 Media, highlights Google's further monopoly on valuable information.