Google’s AI Chatbot Clears US Medical Licensing Exam

Google’s AI-driven medical chatbot, Med-PaLM, has successfully passed a stringent US medical licensing examination, although its responses still fall behind those of human physicians, according to a peer-reviewed study published recently. The launch of ChatGPT, developed by OpenAI and supported by Microsoft, created a competitive atmosphere among tech giants in the rapidly evolving field of AI.

While debates continue over the potential opportunities and challenges of AI, the healthcare sector has already witnessed significant advancements in the technology. AI algorithms have demonstrated their ability to interpret certain medical scans as effectively as human professionals

Google introduced Med-PaLM, its AI solution for responding to medical queries, in a preprint study in December. Unlike ChatGPT, Med-PaLM is not publicly available. Google claims that Med-PaLM is the first large language model to clear the US Medical Licensing Examination (USMLE).

Also Read: Kotli Sattian: Rawalpindi’s Hidden Gem to Be Transformed into a Premier Tourist Destination

The passing score for the USMLE, taken by medical students and physicians in training in the United States, is around 60%. A study in February reported that ChatGPT achieved passing or near-passing scores.

I am very excited that the MedPaLM paper is now published in @Nature https://t.co/BDqshkSIDf

It is a great way to invite the broader scientific community to LLMs. I feel like LLMs are more adopted in the general public than in the scientific community. There are just so many…

— Hyung Won Chung (@hwchung27) July 12, 2023

In a peer-reviewed study published in the Nature Journal, Google researchers revealed that Med-PaLM scored 67.6% on multiple-choice questions based on the USMLE. Google established a new evaluation benchmark to identify and reduce “hallucinations,” where AI models generate incorrect data.

The study’s principal author, Karan Singhal, a Google researcher, reported that the team used the benchmark to evaluate a more recent version of the model, resulting in “super exciting” outcomes. A preprint study released in May, which has not undergone peer review, indicated that Med-PaLM 2 achieved a score of 86.5% on the USMLE, marking an improvement of nearly 20% over its predecessor.