Loading...
Market
Story
lmsys Introduces Open-Source RouteLLM Framework to Cut LLM Costs by Up to 85%
Jul 1, 2024, 05:01 PM
The lmsys team has introduced RouteLLM, a new open-source routing framework designed to reduce the costs of using large language models (LLMs) like GPT-4. By directing simpler queries to cheaper models, RouteLLM claims to achieve cost reductions of over 85% on MT Bench and 45% on MMLU while maintaining 95% of the quality. This initiative is reportedly 40% cheaper than existing routers such as Martian. Developed in collaboration with Anyscale, RouteLLM uses human preference data to intelligently select the best model for each query, ensuring cost-effectiveness without compromising performance.
View original story
Similar markets
Llama 3.1 405B • 25%
Llama 3.1 8B • 25%
Llama 3.1 70B • 25%
Other • 25%
1B • 25%
7B • 25%
72B • 25%
Other • 25%
1B parameter model • 25%
3B parameter model • 25%
11B parameter model • 25%
90B parameter model • 25%
Research & Development • 25%
Customer Service • 25%
Content Creation • 25%
Other • 25%
Claude 3 Haiku • 25%
Opus • 25%
3.5 Sonnet • 25%
Other • 25%
Edge Devices • 25%
Enterprise Solutions • 25%
Multimodal Applications • 25%
Other • 25%
1B • 25%
3B • 25%
11B • 25%
90B • 25%
Natural Language Processing • 25%
Computer Vision • 25%
Recommendation Systems • 25%
Other • 25%
Reduction in hallucinations • 25%
Improvement in factual accuracy • 25%
Enhanced numerical and statistical data integration • 25%
Other • 25%
GPT-4o • 25%
GPT-4o mini • 25%
Other OpenAI models • 25%
Non-OpenAI models • 25%
1B Parameters • 25%
3B Parameters • 25%
11B Parameters • 25%
90B Parameters • 25%
Social media content creation • 25%
Professional video production • 25%
Educational content • 25%
Marketing and advertisements • 25%