Meeting Medical Questions with AI: Introducing Med-PaLM 2

Medical question answering has seen a boost with large language models (LLMs). One such model, Med-PaLM, first surpassed the 'passing' score on US Medical Licensing Exam-style questions. But, long-form answering and real-world scenarios still pose challenges. Enter Med-PaLM 2, an improved version that makes strides in these areas. It combines better base LLMs, fine-tuning for medical domains, and new strategies for reasoning and grounding. Med-PaLM 2 scores up to 86. 5% on the MedQA dataset, a 19% jump from its predecessor. It also shows significant gains on other medical question datasets. Doctors prefer its answers over other physicians' in eight out of nine clinical aspects. In a real-world test, specialists chose Med-PaLM 2's answers over generalists 65% of the time. While specialists' answers were still preferred, both groups found Med-PaLM 2's responses as safe as a doctor's. This highlights its potential in real medical situations.