Official benchmark results publication

Google's Med-Gemini AI Achieves 91.1% Accuracy, Outperforms GPT-4 in Medical Benchmarks

Apr 30, 2024, 03:15 PM

Google has introduced Med-Gemini, a new family of AI models under its Gemini project, specifically designed for medical applications. These models, building on the foundational Gemini 1.0 + 1.5 technology, have been fine-tuned to handle a variety of medical tasks, including answering medical questions with up-to-date knowledge and conversing about medical images and videos. Med-Gemini has achieved state-of-the-art (SoTA) performance on 10 out of 14 benchmarks, surpassing GPT-4 in every benchmark where a direct comparison was made. Notably, Med-Gemini achieved a 91.1% accuracy on the MedQA (USMLE) benchmark using a novel uncertainty-guided search strategy. This development marks a significant advancement in the application of AI in the medical field, resulting from a joint effort with @vivnat.