AMD-powered LUMI Supercomputer Accelerates AI Model Creation for TurkuNLP
TurkuNLP, a leading research group based at the University of Turku in Finland, has harnessed the power of the LUMI supercomputer, equipped with advanced microprocessors from AMD, to create new AI models quickly. The LUMI supercomputer has recently been recognized as the fastest and most energy-efficient supercomputer in Europe.
The demand for large language models (LLMs) that drive generative AI solutions like ChatGPT is on the rise. However, training LLMs requires substantial computational power, and existing models are often proprietary and limited to the English language. TurkuNLP Research Fellow, Sampo Pyysalo, recognized the need to extend the application of LLMs to wider research areas but required a significant performance boost to train models within a reasonable timeframe.
Powered by AMD’s Epyc CPUs and Instinct GPUs, the LUMI supercomputer provides the scale and computational capabilities needed to meet these research demands. In fact, LUMI is two orders of magnitude larger than its predecessors in Finland, offering unprecedented computing capacity. While it used to take the TurkuNLP team six months to pre-train a billion-parameter language model on their previous machines, LUMI can now process around 40 billion tokens, equivalent to characters, syllables, or words, in just two weeks.
Väinö Hatanpää, a machine learning specialist at CSC, expressed excitement about LUMI’s computing capacity, stating that it allows their customers to push the boundaries of machine learning and AI. Pyysalo and his partners in TurkuNLP, Risto Luukkonen and Ville Komulainen, aim to make LLMs more accessible for academic use. They seek practical access to these models, as they believe the current models produced by large multinational corporations are often kept closed to outside use. This led them to create their own models, requiring the use of a supercomputer like LUMI.
Beginning with the Finnish language, which aligns with Turku’s Finnish roots, TurkuNLP leverages advanced AI and machine learning tools in collaboration with Hugging Face. However, building models of this magnitude necessitates significant computational resources, and LUMI has proven to be an invaluable asset in this regard.
LUMI is owned by the EuroHPC Joint Undertaking and was funded jointly by the EuroHPC JU and the LUMI consortium, consisting of ten European countries. The supercomputer is housed in Finland at CSC’s data center, hosted by the LUMI consortium. Notably, LUMI’s GPU partition dwarfs other GPU partitions at CSC, with a staggering 2,560 nodes powered by AMD Epyc processors and 10,240 GPUs.
The collaboration between TurkuNLP and AMD-powered LUMI marks a significant milestone in the advancement of AI research and development. With LUMI’s unparalleled computing capabilities, researchers can accelerate the creation of AI models and expand their applications across various disciplines. This breakthrough reinforces Finland’s position as a leading force in the AI landscape and paves the way for future innovation and collaboration in the field.
