The current trend of using AI models to improve performance is heavily reliant on data. Research recently conducted by Epoch suggests that access to high-quality data will become scarce soon, likely within the next decade. This means that the current development of AI and machine learning will have to take other approaches. Different solutions have already been proposed, such as Joint Empirical Probability Approximation (JEPA) and data augmentation techniques, but none of these provide a seamless, permanent fix to the data issue.
Join top executives from all over the world on July 11-12 in San Francisco to learn more about how organizations can successfully integrate and optimize AI investments to keep technological advancement from slowing down. OpenAI, Microsoft and other organizations will explain the steps and strategies necessary for developing effective AI models that can handle data scarcity and provide solutions thereon.
Yann LeCun, a University of Southern California professor, has proposed an approach of creating more diversified training datasets to ensure a high training quality without the need for data-intensive models. He also suggested that reusing the same data more times could help optimize training, reduce costs and increase the efficacy of the model.
Data augmentation and transfer learning could also be useful in helping AI models handle data scarcity. Data augmentation involves modifying existing data to create synthetic datasets, and transfer learning involves using a pre-trained model and fine-tuning it for a specific task. While both strategies can help in a data constrained environment, they do not solve the problem once and for all.
Ultimately, the truth is, AI models require data, and we are running out of it. The key to finding long-term solutions lies in developing models with the same accuracy and performance which can be trained on a small amount of data and that are robust, interpretable, and explainable. While research and development into such approaches are ongoing, organizations must embrace other techniques, such as the ones mentioned above, to ensure advancement and continued growth.
