fbpx
Breaking: John Snow Labs Achieves State-of-the-Art Medical LLM Accuracy Breaking: John Snow Labs Achieves State-of-the-Art Medical LLM Accuracy
John Snow Labs, a leader in healthcare-related AI, has announced that it set a new benchmark in medical Large Language Models.... Breaking: John Snow Labs Achieves State-of-the-Art Medical LLM Accuracy

John Snow Labs, a leader in healthcare-related AI, has announced that it set a new benchmark in medical Large Language Models. The model achieved state-of-the-art accuracy on the Open Medical LLM leaderboard, beating out hundreds of other models in the process.

In-Person and Virtual Conference

September 5th to 6th, 2024 – London

Featuring 200 hours of content, 90 thought leaders and experts, and 40+ workshops and training sessions, Europe 2024 will keep you up-to-date with the latest topics and tools in everything from machine learning to generative AI and more.

 

If you don’t know, The Open Medical LLM leaderboard evaluates models across nine benchmarks, challenging them to answer thousands of questions from medical licensing exams (MedQA), biomedical research (PubMedQA), and college-level exams in anatomy, genetics, biology, and medicine (MMLU).

So what’s the score? Well according to the company the  Medical LLM has achieved an impressive score of 87.35 on the reproducible test harness of the Open Medical LLM leaderboard. This performance outshines models such as Med-PaLM2, GPT-4, OpenBioLLMLlama, MedLlama, and Orpo-Med, setting a new standard in medical AI accuracy.

Level Up Your AI Expertise! Subscribe Now:  File:Spotify icon.svg - Wikipedia Soundcloud - Free social media icons File:Podcasts (iOS).svg - Wikipedia

The model used is a 7 billion parameter Medical LLM. It not only surpasses all previous models of similar size but is also the first to outperform GPT-4 on the PubMedQA dataset, scoring 78.4 compared to GPT-4’s 75.2. PubMedQA, with its 273,500 questions requiring reasoning over biomedical research texts, now has a new leader in accuracy, matching the single human performance of 78%.

The Medical LLM excels by more than 12 points over all current models of its size while maintaining the capability to run on mobile devices. This advancement is crucial for medical professionals who need to process vast amounts of patient notes efficiently without straining computing resources.

Its accuracy rivals that of larger 8 billion parameter models like BioMistral, achieved just three months ago. This clearly shows that the LLMs are still outpacing even some of the most ambitious predictions of scale and power.

In-Person & Virtual Data Science Conference

October 29th-31st, 2024 – Burlingame, CA

Join us for 300+ hours of expert-led content, featuring hands-on, immersive training sessions, workshops, tutorials, and talks on cutting-edge AI tools and techniques, including our first-ever track devoted to AI Robotics!

 

It’s a great responsibility and honor to provide novel, state-of-the-art, production-ready models to the global healthcare AI community,” said David Talby, CTO of John Snow Labs. “We didn’t give these new models fancy names because we’ll have better ones next week. That’s been the essence of our work for the past seven years, and it’s what makes John Snow Labs the most comprehensive medical language understanding solution on the market.

For more information, visit John Snow Labs.

ODSC Team

ODSC Team

ODSC gathers the attendees, presenters, and companies that are shaping the present and future of data science and AI. ODSC hosts one of the largest gatherings of professional data scientists with major conferences in USA, Europe, and Asia.

1