Loading...
Market
Story
lmsys Introduces Open-Source RouteLLM Framework to Cut LLM Costs by Up to 85%
Jul 1, 2024, 05:01 PM
The lmsys team has introduced RouteLLM, a new open-source routing framework designed to reduce the costs of using large language models (LLMs) like GPT-4. By directing simpler queries to cheaper models, RouteLLM claims to achieve cost reductions of over 85% on MT Bench and 45% on MMLU while maintaining 95% of the quality. This initiative is reportedly 40% cheaper than existing routers such as Martian. Developed in collaboration with Anyscale, RouteLLM uses human preference data to intelligently select the best model for each query, ensuring cost-effectiveness without compromising performance.
View original story
Similar markets
Less than 10% • 25%
10-20% • 25%
20-30% • 25%
More than 30% • 25%
Reduction in hallucinations • 25%
Improvement in factual accuracy • 25%
Enhanced numerical and statistical data integration • 25%
Other • 25%
Llama 3.1 405B • 25%
Llama 3.1 8B • 25%
Llama 3.1 70B • 25%
Other • 25%
Yes • 50%
No • 50%
Less than 50 • 25%
50 to 100 • 25%
101 to 150 • 25%
More than 150 • 25%
Yes • 50%
No • 50%
Above 40% • 25%
30-40% • 25%
20-30% • 25%
Below 20% • 25%
Yes • 50%
No • 50%
Increases significantly • 25%
Increases slightly • 25%
Remains stable • 25%
Decreases • 25%
Less than 10% • 25%
10% to 20% • 25%
20% to 30% • 25%
More than 30% • 25%