Loading...
Market
AI agent achieves Kaggle Grandmaster status using MLE-bench by end of 2024?
3
Resolution / Starting Odds
Yes • 50%
No • 50%
Kaggle leaderboard and announcements
Story
OpenAI Releases MLE-bench with 75 Kaggle Competitions to Evaluate AI Agents' ML Engineering Skills
Oct 10, 2024, 05:33 PM
OpenAI has announced the release of MLE-bench, a new benchmark designed to evaluate the machine learning engineering capabilities of AI agents. The benchmark comprises 75 real-life machine learning engineering competitions sourced from Kaggle. MLE-bench aims to measure how well AI agents perform tasks in machine learning engineering, bridging the gap between theoretical AI knowledge and practical applications in real-world scenarios. The release of this benchmark could accelerate the development of AI agents capable of writing machine learning code, potentially leading to self-improving AI systems. The benchmark raises the prospect of AI agents achieving Kaggle Grandmaster status in the future.
View original story
Similar markets
Yes, it will outperform AdamW • 25%
Yes, it will outperform SGD • 25%
Yes, it will outperform AdaGrad • 25%
No, it will not outperform any • 25%
Yes • 50%
No • 50%
Yes • 50%
No • 50%
GLUE • 25%
SuperGLUE • 25%
SQuAD • 25%
Other • 25%
OpenAI o1 • 25%
GPT-4 • 25%
Gemini • 25%
Claude • 25%
Llama 3.1 405B • 25%
GPT-4o • 25%
Claude Sonnet 3.5 • 25%
Other • 25%
