ChatGPT Achieves Success as Virtual AI Radiologist in Radiology Board Exam

Date:

A recent study published in the Radiological Society of North America’s journal has shown that ChatGPT, an AI chatbot developed for language interpretation and response generation, has successfully passed a radiology board-style exam. This demonstrates both the potential and the limitations of the current technology.

Lead author Rajesh Bhayana, MD, FRCPC, an abdominal radiologist and technology lead at University Medical Imaging in Toronto, Canada, said, “The use of large language models like ChatGPT is exploding and only going to increase. Our research provides insight into ChatGPT’s performance in a radiology context, highlighting the incredible potential of large language models, along with the current limitations that make it unreliable.”

ChatGPT was tested using 150 multiple-choice questions designed to match the style, content, and difficulty of the Canadian Royal College and American Board of Radiology exams. It was found that GPT-3.5 model had an accuracy of 69%, which just passed the threshold of 70% set by the Royal College. Further, the model performed relatively well on questions requiring lower-order thinking (84%), but struggled with higher-order thinking questions (60%).

In a follow-up study, GPT-4 model, the improved version of GPT-3.5 was tested, and it showed better accuracy of 81%, outperforming GPT-3.5 and scoring above the passing threshold of 70%. It was also observed that GPT-4 performed much better than GPT-3.5 on higher-order thinking questions (e.g. questions involving description of imaging findings and application of concepts).

The researchers found that, although considerably improved, ChatGPT can still generate incorrect responses, which they termed as “hallucinations”. This is particularly dangerous if solely relied on for information. Hence, the researchers cautioned that ChatGPT should not be used for quick information recall and should only be used to spark ideas.

See also  Growing Fear of AI Causing Unemployment as ChatGPT is Accused of Wiping Out Jobs

The company ChatGPT is owned by OpenAI, a research laboratory based in California and was recently declared as the fastest growing consumer application in history. It is increasingly being incorporated into popular search engines used to search for medical information.

Rajesh Bhayana is an abdominal radiologist and technology lead at University Medical Imaging Toronto, Toronto General Hospital in Toronto, Canada. He is the lead author of the two studies published in Radiology, a journal of the Radiological Society of North America (RSNA). He is skilled in medical imaging informatics, healthcare innovation, medical education, and research.

Frequently Asked Questions (FAQs) Related to the Above News

Please note that the FAQs provided on this page are based on the news article published. While we strive to provide accurate and up-to-date information, it is always recommended to consult relevant authorities or professionals before making any decisions or taking action based on the FAQs or the news article.

Share post:

Subscribe

Popular

More like this
Related

Obama’s Techno-Optimism Shifts as Democrats Navigate Changing Tech Landscape

Explore the evolution of tech policy from Obama's optimism to Harris's vision at the Democratic National Convention. What's next for Democrats in tech?

Tech Evolution: From Obama’s Optimism to Harris’s Vision

Explore the evolution of tech policy from Obama's optimism to Harris's vision at the Democratic National Convention. What's next for Democrats in tech?

Tonix Pharmaceuticals TNXP Shares Fall 14.61% After Q2 Earnings Report

Tonix Pharmaceuticals TNXP shares decline 14.61% post-Q2 earnings report. Evaluate investment strategy based on company updates and market dynamics.

The Future of Good Jobs: Why College Degrees are Essential through 2031

Discover the future of good jobs through 2031 and why college degrees are essential. Learn more about job projections and AI's influence.