AI Feedback Loop: Researchers Warn of Model Collapse as AI Trains on AI-generated Content

Researchers have found that models trained on AI-generated data tend to collapse over time, meaning that they forget the true underlying data distribution. The team discovered that these defects were irreversible, and models quickly forgot most of the original data. The result of AI training on AI-generated data is that models deteriorate and produce less non-erroneous variety in their responses, increasing the likelihood of discrimination, particularly against minority groups. The researchers suggested avoiding contamination with AI-generated data to prevent model collapse, and instead periodically retrain or refresh the model entirely with predominantly human-produced data. The findings underscore the risks of unchecked generative processes in AI and guide future research to manage model collapse and maintain the integrity of generative models over time.