Sarvam AI Launches Open-Source AI Model for Hindi Language Innovation
Sarvam AI, an Indian tech company, has introduced OpenHathi-Hi-0.1, an advanced Hindi language tool that is freely accessible to all. With the aim of enhancing Hindi AI, Sarvam AI has made this tool available to everyone, acting as a catalyst for further advancements in Hindi technology.
The foundation of OpenHathi-Hi-0.1 is built upon Meta AI’s Llama 2-7B model, establishing itself as a formidable contender comparable to GPT-3.5 for Indic languages. In their blog, Sarvam AI discussed the challenges they faced in developing this language tool. They emphasized the importance of tokenization, a crucial component in ensuring the effectiveness of text tools. Tokenization proved to be more resource-intensive for Hindi compared to English due to the limited availability of training text in the Hindi language. However, the Sarvam AI team successfully addressed this issue through two steps, making the process more efficient and cost-effective.
To evaluate the model’s performance, Sarvam AI conducted tests involving translation and sentiment analysis.
The commendable step taken by Sarvam AI is making the base model accessible on the Hugging Face platform. This enables developers to enhance the model’s capabilities for specific tasks, fostering collaborative efforts to improve AI for everyone.
Co-founders Pratyush Kumar and Vivek Raghavan, who had previous experience at AI4Bharat, joined forces with Sarvam AI to leverage the language resources and benchmarks from AI4Bharat for training OpenHathi.
With a team of around 18 individuals, Sarvam AI aims to develop voice-activated computers that can comprehend Indian accents effectively. Their mission is to ensure that technology seamlessly aligns with the diverse range of languages spoken across India.
Recently, Sarvam AI achieved a major funding milestone, securing millions in Series A funding. The funding was led by Lightspeed Ventures, with Peak XV and Khosla Ventures also participating. This injection of capital positions the startup for further expansion. Alongside OpenHathi-Hi-0.1, Sarvam AI is actively working on a suite of enterprise-grade models through its full-stack Generative AI platform, which is expected to be unveiled soon.
In summary, Sarvam AI has launched OpenHathi-Hi-0.1, an open-source AI model for Hindi language innovation. By sharing this powerful language tool with the public, Sarvam AI hopes to contribute to the advancement of Hindi AI. With their recent funding success and ongoing projects, Sarvam AI is well-positioned to make significant strides in the field of AI technology.
