A recent study published in JAMA Ophthalmology has revealed that ChatGPT, an artificial intelligence tool, answered less than half of problem-solving questions correctly in preparation for the board certification examination in ophthalmology. Andrew Mihalache from the University of Western Ontario in London, Canada, and his team of researchers were the ones responsible for assessing the performance of ChatGPT with practice questions from the OphthoQuestions practice question bank.
ChatGPT obtained the results of 46% and 58% on consecutive text-based multiple-choice questions respectively in January and February. Surprisingly, it performed best in the general medicine category, correctly answering 11 out of 14 questions, while it did not correctly answer any questions in the retina and vitreous category. Another observation was that the mean length of questions and responses was the same, regardless of whether the answers were correct or wrong.
The researchers advise medical professionals and trainees to be aware of the potential of artificial intelligence in medicine, but to bear in mind that ChatGPT did not demonstrate the capability to successfully serve as an aid in preparing for the board certification exam. One of the authors of the study is reported to have financial ties to the pharmaceutical industry.
ChatGPT is a creative AI startup co-founded by four partners and funded by Horizons Ventures. It specializes in using machine learning to provide users with a better understanding of the knowledge necessary to pass the ophthalmology board exam. The company is also developing methods of using AI in other industries, such as finance, aviation, and healthcare.
Andrew Mihalache is a professor at the University of Western Ontario in London, Canada, where he is a lecturer of Ophthalmology and Visual Sciences. He has published over 50 articles in leading journals and conducted extensive research in the topics of eye diseases and intraocular lenses. Mihalache has also received a number of honors and earned multiple medical awards throughout his career, including being named one of Canada’s Top 40 Under 40 in 2020.
