Researchers at the Feinstein Institutes for Medical Research issued a warning regarding the use of the artificial intelligence tool ChatGPT for medical education in gastroenterology, following its dismal performance on an ACG multiple-choice, self-assessment test. This test requires a score of 70% to pass. ChatGPT versions 3 and 4, which generate human-like responses based on user prompts, were unable to achieve even close to that score, with a final score of 65.1% and 62.4%, respectively.
Arvind Trindade, MD, distinguished Regional Director of Endoscopy at Northwell Health System and Associate Professor of Medicine at the Feinstein Institute, commented on this incident. He said that ChatGPT is not capable of grasping the current issues, and its primary capability is to forecast the next word in a specific string of text in an attempt to generate an anticipated response, no matter whether such a response is accurate or not. Therefore, it is perilous when related to giving medical advice or education.
To make ChatGPT a potentially useful tool for edification or patient utilization in gastroenterology, Dr. Trindade mentioned that it needs to have access to updated sources like gastroenterology guidelines, medical journals, and databases. This is because its answers are generated from existing information. Nevertheless, regardless of how much training it may receive, it may never be able to completely replace the certified medical training required in this field. It may, however, be a helpful tool in the future.
The Feinstein Institutes for Medical Research is a center of biomedical research and education part of the Northwell Health that specializes in diseases, conditions, and health issues related to the field of Gastroenterology. This is undertaken by a team of researchers, physicians, and students.
Dr. Trindade is an internationally renowned gastroenterologist and endoscopist who has published several seminal articles on endoscopic procedures. He is a specialist in the field of endoscopy and serves as an Associate Professor of Medicine at the Feinstein Institute. He is also the distinguished Regional Director of Endoscopy at Northwell Health System.
