Loading...
Market
Will future ARC-AGI iterations achieve 100% solvability by end of 2025?
3
Resolution / Starting Odds
Yes • 50%
No • 50%
Official announcements from ARC-AGI organizers
Story
Independent NYU Study Finds 98.7% of ARC-AGI Tasks Solvable by Humans Ahead of November 10 Competition Deadline
Sep 4, 2024, 05:53 PM
Researchers at New York University (NYU) conducted an independent study on the ARC-AGI tasks, revealing that 98.7% of the public tasks are solvable by humans. The study found that 790 out of 800 tasks could be completed by at least one Mechanical Turk worker. This finding underscores the gap between human and AI performance on these tasks. The ARC-AGI competition, which challenges participants to develop AI capable of solving these tasks, will end on November 10, 2024. Researchers aim for future iterations to achieve 100% solvability and to establish human baselines on the private test set. Many high-scoring entries in the competition currently rely on basic brute-force program search.
View original story
Similar markets
Google DeepMind • 25%
Anthropic • 25%
Meta AI • 25%
Other • 25%
o3 remains the top performer • 25%
Another model surpasses o3 • 25%
o3 ties with another model • 25%
No new models tested • 25%
Yes • 33%
No • 33%
Unclear/Disputed • 34%
Technological leadership • 25%
National security • 25%
Economic growth • 25%
Other • 25%
Below 85% • 25%
85% to 90% • 25%
90% to 95% • 25%
Above 95% • 25%
Level 1 • 25%
Level 2 • 25%
Level 3 • 25%
Level 4 or higher • 25%
OpenAI • 25%
Google • 25%
DeepMind • 25%
Other • 25%