Researchers from the University of Maryland School of Medicine (UMSOM) have conducted a study to see whether the newly available ChatGPT, an artificial intelligence (AI) chatbot, provides accurate and reliable health advice. The findings, published in the journal Radiology, show that the answers generated are appropriate 88% of the time, but can still provide outdated information or even fictitious answers.
The study assessed 25 questions related to breast cancer screening and posed each question to ChatGPT three times. Three fellowship-trained radiologists evaluated the responses and found that they were accurate 22 out of 25 times.
Dr. Paul Yi, the corresponding author of the study and Director of the UM Medical Intelligent Imaging Center (UM2ii) said, “We found ChatGPT answered questions correctly about 88 percent of the time, which is pretty amazing.” This AI chatbot is known for summarizing information into easy-to-understand responses for consumers, making it incredibly convenient for anyone seeking health advice.
However, the study does indicate some limitations with ChatGPT, such as providing only one set of recommendations on breast cancer screening, being unaware of alternative recommendations from entities such as the Centers for Disease Control, or giving inconsistent responses to certain questions. Dr. Paul Yi also remarked that “Consumers should be aware that these are new, unproven technologies, and should still rely on their doctor, rather than ChatGPT, for advice.”
The University of Maryland School of Medicine recently launched plans for an Institute for Health Computing, which aims to leverage AI and computing to create a learning healthcare system. The vision is to use these technologies to enhance patient care and provide population health services.
The study was funded by the individual institutions, with researchers from the University of Maryland Medical Center, the Massachusetts General Hospital, and the Johns Hopkins University School of Medicine contributing.
To lead this new era of healthcare, Dean Mark T. Gladwin, Dean of UMSOM and Vice President for Medical Affairs of University of Maryland, Baltimore, said the School of Medicine needs to embrace the power of clinical analytics and precision medicine to improve patient care.
This news release was issued by Deborah Kotz, Senior Director of Media Relations for the University of Maryland School of Medicine. UMSOM is now in its third century and was chartered in 1807 as the first public medical school in the United States. It is one of the fastest growing, top-tier biomedical research enterprises in the world, and so far, has a faculty of more than 3,000 physicians, scientists, and allied health professionals. With an annual budget of over $6 billion and economic impact of nearly $20 billion, the School of Medicine works closely in partnership with the University of Maryland Medical Center and Medical System to provide research-intensive, academic, and clinically based care for nearly 2 million patients each year.
