In September 2023, the incorporation of image analysis capabilities in ChatGPT by OpenAI sparked interest in its potential application for diagnosing skin lesions, including melanoma through dermatoscopic images. Researchers recently conducted a study to evaluate the performance of ChatGPT Vision in identifying melanoma from dermoscopic images.
The study involved 100 melanocytic lesions, including 50 melanomas and 50 non-atypical benign nevi, sourced from the International Skin Imaging Collaboration database. ChatGPT Vision was tasked with providing three differential diagnoses ranked in order of likelihood.
The results revealed that ChatGPT Vision exhibited a sensitivity of 32%, specificity of 40%, and an overall diagnostic accuracy of 36% for the initial diagnosis. However, when considering the top three differential diagnoses proposed by the chatbot, the sensitivity increased to 56%, with a specificity and precision of 53% and 55%, respectively.
When differentiating between malignant and benign lesions, the sensitivity, specificity, and precision were calculated at 46%, 78%, and 62%, respectively, for the most probable diagnosis, and 78%, 47%, and 62% for the top three diagnoses suggested by ChatGPT.
The study highlighted that while ChatGPT Vision could potentially assist in describing images, its performance in diagnosing melanoma from dermatoscopic images fell short compared to established artificial intelligence algorithms. The researchers emphasized that the risk of misclassification and the potential to miss melanomas remain significant concerns, limiting the chatbot’s suitability for clinical use.
The limitations of the study, such as the small sample size, absence of dysplastic lesions, and oversight of crucial clinical data, underline the need for further refinement and validation of ChatGPT Vision’s capabilities in diagnosing melanoma. The study’s findings suggest that while ChatGPT may have potential, its current effectiveness in diagnosing melanoma is not yet at a standard suitable for clinical application.
