Official announcements from Scale AI or updates on the SEAL Leaderboards platform

Scale AI Launches SEAL Leaderboards for Expert-Driven LLM Evaluations, Platform in GA

May 29, 2024, 04:55 PM

Scale AI has launched the SEAL Leaderboards, a new ranking system for frontier large language models (LLMs). This initiative is designed to provide expert-driven, trustworthy evaluations using private datasets that cannot be manipulated. The leaderboards are continuously updated with new data and models, and domain experts handle the evaluations. Scale AI aims to address the issues with current LLM evaluations, which often show discrepancies between qualitative experiences and quantitative rankings. The SEAL Leaderboards are seen as a serious contender to existing evaluation systems like lmsys.org. Scale AI is also taking the Scale Evaluation platform into General Availability (GA) and encourages model submissions via contact at seal@scale.com.