SEAL Leaderboards on Scale AI's official website

Scale AI Launches SEAL Leaderboards for LLMs in Coding, Math, Instruction, and Spanish

May 29, 2024, 05:42 PM

Scale AI has launched the SEAL Leaderboards, a new evaluation platform for large language models (LLMs). The leaderboards use private datasets and expert evaluations to rank LLMs in various domains, including coding, math, instruction following, and Spanish. This initiative aims to address common issues in model evaluations, such as data contamination and rater quality. The SEAL Leaderboards are continuously updated with new data and models, and Scale AI is inviting model developers to participate. The platform also extends GSM1K to various domains and uses ELO-scale rankings via the Bradley-Terry method. The launch has been well-received, with industry experts highlighting its potential to provide more trustworthy and accurate assessments of LLMs.