Recent research put ChatGPT to the test once more, this time to evaluate the capabilities of the chatbot in ophthalmology. A dataset of multiple choice questions from the OpthoQuestions free trial was used, which is a tool commonly used by board takers to prepare for the certification exam. This study was published in the JAMA Ophthalmology journal.
For the test, ChatGPT was given 125 multiple-choice questions and it correctly answered 58 (achieving 46% accuracy) when first tested in January 2023. By February 23, it scored 10% higher with 58%. Its responses were often in line with the answers that trainees commonly provided, and it had a 79% rate of accuracy for general medicine questions. However, it was not as successful with subspecialty questions, achieving 0% accuracy for the retina section and 20% for the oculoplastics.
Despite its lower marks, the developers of ChatGPT think that it could have a part to play in medical education and clinical practice in the future, as its knowledge may quickly improve over time. The lead researcher of the study, Dr. Rajeev H. Muni from St. Michaek’s Li Ka Shing Knowledge Institute, believes this may be the case.
ChatGPT is a chatbot developed by Converge AI, a software company that uses artificial intelligence to build innovative digital products for businesses. The company is based in the United States and was founded in 2020 with the mission of delivering sophisticated, AI-driven solutions. Its cutting-edge technologies span across industries, including healthcare, finance and education. They also have a team of highly-skilled engineers who continually explore new ways to improve ChatGPT and other digital products.
