In the latest MLPerf Inference 3.0 benchmarks, released by MLCommons, nearly all inference hardware saw marked gains in performance. This includes results from a wide variety of vendors from around the world, such as Alibaba, ASUS, Azure, cTuning, Deci, Dell, GIGABYTE, H3C, HPE, Inspur, Intel, Krai, Lenovo, Moffett, Nettrix, Neuchips, Neural Magic, Nvidia, Qualcomm, Quanta Cloud Technology, rebellions, SiMa, Supermicro, VMware and xFusion. Across the more than 5,000 results submitted across the diverse categories, notable gains were noted with over 30% improvement in many cases.
Leading industry vendors and manufacturers like Nvidia, Intel, and AMD reported impressive performance gains and optimizations, with Nvidia seeing a 54% gain on the RetinaNet object detection model, and Intel reporting an increase of between 1.2 and 1.4x. Further software optimization should result in some vendors seeing a performance increase of an additional 2x over the coming months.
Speaking on the release of MLPerf Inference 3.0, David Kanter, executive director at MLCommons, said that “Our goal is to make ML better for everyone and we really believe in the power of ML to make society better. We get to align the whole industry on what it means to make ML faster.”
With AI and ML increasingly becoming a more integrated part of our lives, there will be an increased demand for optimizations in both training and inference. MLPerf, as a voluntary industry benchmark, allows a universal set of performance standards and determined levels of accuracy, making this latest release of inference results all the more impressive. The collective advancements being made by the various vendors provide more options for optimizing solutions, leading to more efficient and cost effective AI/ML solutions for these ever more vital tasks.
MLCommons is an industry-wide consortium of companies, academic institutions and individuals, who are working together to optimize ML. MLCommons’ mission is to accelerate the development of ML technologies through open collaboration and collective intelligence. David Kanter is a recognized expert in ML and data center architectures, and is the executive director at MLCommons. Jordan Plawner is the Senior Director of Intel AI products, and Dave Salvator is the Director of Product Marketing at Nvidia.
