ChatGPT’s ability to accurately recommend breast cancer screening tests has been hailed as impressive by researchers. In a study published in the Journal of the American College of Radiology, ChatGPT models 3.5 and 4 were tested on their adherence to breast cancer and breast pain screening guidelines. The models achieved impressive accuracy rates, with ChatGPT 4 scoring 98.4% for breast cancer screening recommendations and 77.7% for breast pain screening recommendations. Meanwhile, ChatGPT 3.5 achieved accuracy rates of 88.9% and 58.3% for breast cancer and breast pain screening, respectively.
The researchers, led by Marc D. Succi, MD, and Arya Rao, BA, from Mass General Brigham Radiology and Harvard Medical School, aimed to highlight the potential of artificial intelligence (AI) in enhancing and supporting clinical decision-making. They believe that integrating an AI-based tool into existing healthcare systems could significantly improve efficiency by utilizing the vast amount of available patient information and medical records.
The study involved presenting 21 breast cancer or breast pain prompts to the ChatGPT models and comparing their responses to the guidance provided by the American College of Radiology’s Appropriateness Criteria. The high accuracy rates achieved by the models indicate their potential to act as trained consultants, bridging the gap between healthcare professionals and radiologists to recommend the most appropriate imaging tests without delay.
Dr. Succi sees AI technology like ChatGPT as a valuable tool that can help reduce burnout and administrative tasks for physicians. By providing clinical decision support for noninterpretive tasks, such as test ordering, these AI models can free up primary care doctors and radiologists to focus more on patient care. For example, when seeing a patient who may benefit from breast cancer screening, doctors can use ChatGPT to promptly suggest the appropriate imaging test based on the patient’s age and demographics. This not only saves time but also ensures important tests are not overlooked.
While the findings of the study are promising, there are still limitations and potential downsides to consider. Privacy and bias are among the main concerns when using AI technology for imaging. Protecting patient data and minimizing bias in recommendations are challenges that need to be addressed. Additionally, the study highlights that ChatGPT models are generalist models trained on various data sources and the internet, rather than being specific to medicine or subspecialties. However, as more fine-tuned models specific to medicine become available, the performance and usefulness of AI tools like ChatGPT are expected to improve further.
In conclusion, the study demonstrates the impressive ability of ChatGPT models to accurately recommend breast cancer screening tests. While further research and development are needed to address privacy, bias, and other limitations, the potential of AI in healthcare decision-making is promising. By augmenting the work of healthcare professionals, AI tools like ChatGPT have the potential to enhance efficiency and allow doctors to focus more on patient care. As the field of AI continues to evolve, it is expected to play an increasingly important role in healthcare decision support.
