MLPerf 3.0 Benchmark Introduces LLMs, Demonstrating Significant Increase in AI Training Performance

MLCommons has announced the latest set of results for its MLPerf training 3.0 benchmark, aimed at providing an industry standard set of measurements for machine learning (ML) model training performance. The latest round of training benchmarks includes over 250 different performance results from 16 vendors, with a significant boost in performance across the board, revealing how ML capabilities are outpacing Moore’s Law. The most recent results demonstrate a rise in performance of between 5% and 54% over the past year alone, which executive director David Kanter, described as incredible and about 10X faster than Moore’s Law. Major factors contributing to ML training include improved hardware, algorithms, and software, as well as larger and more efficient systems. The latest benchmark also introduced Large Language Model (LLMs) testing, with GPT-3 as the first focus. The test is highly demanding, requiring vendors to push their silicon to its limits, and Nvidia and CoreWeave broke records on the benchmarking process across multiple workloads.