LiveCodeBench official results or Scale AI's official reports

Scale AI's PlanSearch Enhances Claude 3.5 LLM Code Generation with SOTA Method

Sep 9, 2024, 01:16 PM

Scale AI has proposed a new method called PlanSearch to enhance the diversity and efficiency of large language model (LLM) code generation. This novel search algorithm significantly improves the performance of Claude 3.5, achieving a pass@200 of 77.0% on LiveCodeBench, compared to a pass@1 of 41.4% without search. The method aims to address the challenges of scaling inference capabilities for optimal performance in LLMs. PlanSearch is a state-of-the-art (SOTA) test-time compute method. This development is part of broader efforts to improve LLMs, which are reshaping interactions with technology through applications such as AI-powered chatbots and complex language understanding tasks.