Microsoft Launches Robust AI ‘Small Language Model’ for Researchers
Microsoft has released its newest compact small language model titled Phi-2 that continues to perform at par or better than certain larger open-source Llama 2 models with less than 13 billion parameters. Over the past few months, the Machine Learning Foundations team at Microsoft Research has released a suite of small language models (SLMs) called Phi that achieve remarkable performance on a variety of benchmarks.
The first model, the 1.3 billion parameter Phi-1 achieved state-of-the-art performance on Python coding among existing SLMs (specifically on the HumanEval and MBPP benchmarks).
We are now releasing Phi-2, a 2.7 billion-parameter language model that demonstrates outstanding reasoning and language understanding capabilities, showcasing state-of-the-art performance among base language models with less than 13 billion parameters, the company said in an update.
Phi-2 is an ideal playground for researchers, including for exploration around mechanistic interpretability, safety improvements, or fine-tuning experimentation on a variety of tasks. We have made Phi-2 available in the Azure AI Studio model catalog to foster research and development on language models, said Microsoft.
The massive increase in the size of language models to hundreds of billions of parameters has unlocked a host of emerging capabilities that have redefined the landscape of natural language processing.
However, a question remains whether such emergent abilities can be achieved at a smaller scale using strategic choices for training, e.g., data selection. Our line of work with the Phi models aims to answer this question by training SLMs that achieve performance on par with models of much higher scale (yet still far from the frontier models), said Microsoft.
The company has also performed extensive testing on commonly used prompts from the research community. We observed a behavior in accordance with the expectation we had given the benchmark results, said the tech giant.
With Phi-2, Microsoft aims to provide researchers with a powerful tool to delve into language models and explore various avenues of improvement and innovation. By making the model available in Azure AI Studio, Microsoft is actively encouraging research and development in this field.
The Phi models have proven their mettle in achieving state-of-the-art performance and showcasing outstanding reasoning and language understanding capabilities. This signifies a significant advancement in the field of natural language processing and opens up new possibilities for researchers to explore.
As the landscape of language models continues to evolve, Microsoft remains at the forefront, pushing boundaries and setting new benchmarks. With Phi-2, researchers have a versatile and robust tool to drive innovation and make further strides in the realm of language understanding.
Microsoft’s dedication to advancing AI and language models is clear, as they continue to release cutting-edge models that outperform previous versions and even larger open-source models. The Phi-2 language model stands as a testament to their commitment to providing the research community with effective tools for exploration and development.
This latest release by Microsoft promises to revolutionize the understanding and application of language models in various domains. Researchers now have the opportunity to harness the power of Phi-2 and delve into uncharted territories, nurturing groundbreaking advancements in natural language processing.
