Official report or publication by CAIS

AI Bot FiveThirtyNine Achieves 87.7% Accuracy in Forecasting with GPT-4o

Sep 9, 2024, 10:49 PM

The Center for AI Safety (CAIS) has developed an AI forecasting bot named FiveThirtyNine, built on GPT-4o, which claims to perform superhumanly well in predicting various events. The bot achieved an 87.7% accuracy rate on 177 predictions. This development, highlighted by DanHendrycks, is part of a broader trend where AI systems are showing superhuman capabilities in forecasting and persuasion, on par with groups of human forecasters. Demonstrations and previously published papers indicate that AI forecasters can automate most prediction markets and outperform human forecasters when used in conjunction with them. However, individual large language models (LLMs) alone are not as effective as crowds of AI or human forecasters.