Microsoft’s compact Phi-2 rivals larger AI models
Microsoft has introduced a new small language model (SML) called Phi-2, which aims to match the capabilities of larger artificial intelligence (AI) models. Despite being trained on just 2.7 billion parameters, Phi-2 has performed impressively in various benchmarks, outperforming base language models with fewer parameters. The announcement was made during Microsoft’s Ignite 2023 event, where CEO Satya Nadella highlighted Phi-2’s competence in critical reasoning, language understanding, and other academic benchmarks comparable to large language models (LLMs).
Phi-2 has demonstrated proficiency in math, coding, and common sense reasoning tests, surpassing models like Mistral and Llama 2. In fact, it even outperformed Google’s Gemini Nano 2 on multi-step reasoning tasks. Microsoft Research conducted tests using proprietary datasets as well as commonly used prompts in research circles. The success of Phi-2 is attributed to advances in data curation techniques and high-quality data.
Microsoft Research believes that the compact size of Phi-2 positions it as a frontrunner in AI and machine learning research. They see it as an ideal tool for exploration in areas such as mechanistic interpretability, safety improvements, and fine-tuning experimentation across various tasks. Prior to Phi-2, Microsoft Research developed Phi-1, a 1.3 billion parameter SML, and Phi-1.5, which included advanced common sense reasoning.
In addition to AI model development, Microsoft is also venturing into custom chip development. The company has hinted at integrating custom chips like Maia and Cobalt to compete against rivals like Google in the AI space. Microsoft has recently announced partnerships with the governments of Australia and the UK to improve infrastructure for AI and other emerging technologies. However, the long-term collaboration with ChatGPT maker OpenAI has raised concerns among antitrust regulators worldwide.
In summary, Microsoft’s introduction of Phi-2, a compact SML that rivals larger AI models, has showcased impressive performance in various benchmarks. With its proficiency in critical reasoning, language understanding, and other tests, Phi-2 has the potential to play a leading role in AI and machine learning research. Microsoft’s entry into the AI sector extends beyond model development, as the company explores custom chip development and forms strategic partnerships to advance emerging technologies.
