Jeff Bezos-Backed Perplexity AI Faces Scrutiny Over Ignoring Robots.txt Amid SoftBank Partnership

Jun 19, 2024, 01:24 PM

Perplexity AI, an AI-powered search engine backed by Jeff Bezos, is facing scrutiny over allegations of unethical practices. Reports indicate that Perplexity AI has been found to provide inaccurate information about its user agent and is accused of ignoring robots.txt files, which are used by websites to manage crawler access. Investigations by Wired and other sources suggest that Perplexity is using surreptitious methods to scrape content, raising questions about its transparency and adherence to web standards. Despite these controversies, Perplexity AI has announced a strategic partnership with SoftBank in April to expand its user base in Japan and take on Google. As part of this deal, SoftBank customers will receive one year of Perplexity Pro for free.