Google’s AI research institute Google DeepMind has unveiled a groundbreaking new training technique that promises to revolutionize the way AI models are trained. The innovative method, known as JEST (Joint Example Selection), is said to offer a significant boost in both training speed and energy efficiency, outperforming alternative methods by a whopping 13 times in terms of performance and 10 times in power efficiency.
Traditionally, AI model training has focused on individual data points, but JEST takes a different approach by training based on entire batches. The technique involves creating a small AI model that evaluates data quality from a high-quality source, ranking batches by quality, and then training a larger model based on the findings of the small model. By directing the data selection process towards well-curated datasets, JEST has demonstrated superior performance compared to state-of-the-art models.
The research paper published by DeepMind highlights the critical importance of high-quality training data for the success of the JEST method. The method’s reliance on expert-level research skills and human-curated data sets sets it apart from other techniques, making it more challenging for amateur AI developers to achieve similar results.
The timing of DeepMind’s research is significant as the tech industry and governments worldwide are increasingly concerned about the environmental impact of AI data centers. With AI workloads projected to consume a massive amount of power, innovations like JEST offer a glimpse into more energy-efficient training methods that could help reduce costs and environmental impact.
While the widespread adoption of the JEST method remains to be seen, it has the potential to address the escalating power demands of AI training and mitigate the environmental concerns associated with it. As the debate between cost reduction and output scale continues, the future of AI training methods could be shaped by groundbreaking techniques like JEST.
