A recent study published in JAMA reveals that generative AI can accurately diagnose complex medical cases. The study involved testing the diagnostic ability of Chat-GPT 4, a famous publicly available chatbot, using clinicopathological case conferences. The AI achieved promising results, correctly identifying the top diagnosis nearly 40% of the time and providing the correct diagnosis in its list of potential diagnoses in 64% of challenging cases.
Generative AI is a kind of artificial intelligence that focuses on creating new content rather than processing and analyzing present data. Chatbots are well-known examples of generative AI, as they use natural language processing (NLP) to understand, generate, and interpret human-like language. However, there is much ambiguity about the performance of generative AI chatbots in the medical field, particularly in handling complex diagnostic reasoning.
To examine the chatbot’s diagnostic skills, physician-researchers at Beth Israel Deaconess Medical Center (BIDMC) used 70 complex and challenging patient cases, including laboratory data, imaging studies, and histopathological findings, published in the New England Journal of Medicine for educational purposes. According to Adam Rodman, MD, MPH, co-director of the Innovations in Media and Education Delivery (iMED) Initiative at BIDMC and an instructor in medicine at Harvard Medical School, Recent advances in artificial intelligence have led to generative AI models that are capable of detailed text-based responses that score highly in standardized medical examinations. We wanted to know if such a generative model could ‘think’ like a doctor, so we asked one to solve standardized complex diagnostic cases used for educational purposes. It did really, really well.
Although chatbots cannot replace a trained medical professional’s knowledge and skills, the study’s findings suggest that generative AI could be a promising adjunct to human cognition in diagnosis. It has the potential to help physicians make sense of complex medical data, broaden or refine their diagnostic thinking, and transform healthcare delivery. However, the researchers emphasize the need for more research to address the benefits, optimal use, and limitations of such technology.
