Italian researchers have discovered that ChatGPT, the popular language model developed by OpenAI, has the ability to generate persuasive yet fictitious medical data, raising concerns about the potential misuse of artificial intelligence in medical research, as reported in Tech Xplore.
Giuseppe Giannaccare, an eye surgeon at the University of Cagliari, led the research team that found ChatGPT, specifically the GPT-4 version, capable of creating a fake dataset supporting one surgical eye procedure over another in a matter of minutes.
This discovery sheds light on the potential darker side of AI, highlighting the ease with which AI models can generate and manipulate data to produce biased results and false medical evidence.
The implications of such capabilities extend beyond academic nuisances or financial penalties. In the medical field, where decisions are made based on research outcomes, the potential for AI-generated false data to influence medical procedures is a serious concern.
It was one thing that generative AI could be used to generate texts that would not be detectable using plagiarism software, but the capacity to create fake but realistic data sets is a next level of worry, said Elisabeth Bik, a research-integrity consultant in San Francisco.
It will make it very easy for any researcher or group of researchers to create fake measurements on non-existent patients, fake answers to questionnaires or to generate a large dataset on animal experiments.
The study, titled Large Language Model Advanced Data Analysis Abuse to Create a Fake Data Set in Medical Research, acknowledges that closer scrutiny could reveal signs of possible fabrication in the data.
The researchers also emphasize the need for the scientific community to develop better approaches for fraud detection in response to the potential misuses and threats connected to AI.
Giannaccare underscores the dual nature of AI, acknowledging its potential benefits for scientific research while cautioning against its misuse.
The discovery prompts reflections on how the scientific community will address the challenges posed by AI-generated data and maintain academic integrity in the face of evolving technologies.
The findings of the research team were published in the journal JAMA Ophthalmology.
