China’s search engine pioneer, Sogou founder Wang Xiaochuan, recently tweeted about the need for China to have its own version of OpenAI. In response to this, Chinese entrepreneur Baichuan Intelligence has launched its own large language model, Baichuan-13B, which is set to rival OpenAI.
Baichuan’s founder, Wang Xiaochuan, is highly regarded in the field of computer science, having been a prodigy at Tsinghua University and the founder of Sogou, which was later acquired by Tencent. This background positions Baichuan as one of China’s most promising developers of large language models (LLMs).
After leaving Sogou in late 2021, Wang launched Baichuan and received $50 million in funding from angel investors. Their first LLM, Baichuan-7B, was released in June, followed by the foundational model, Baichuan-13B. Both models are open-source and optimized for commercial use.
Baichuan-13B is a 13 billion-parameter model based on the Transformer architecture, similar to OpenAI’s GPT model. It has been trained on a mixture of Chinese and English data, like other Chinese LLMs. The impressive number of parameters enables the model to generate and analyze text effectively. In comparison, Meta’s LLaMa model uses 1 trillion tokens with 13 billion parameters.
One of the remarkable aspects of Baichuan-13B is its compatibility with consumer-grade hardware, such as Nvidia’s 3090 graphic cards. This is notable considering the AI chip sanctions imposed on China by the U.S., as it ensures that Baichuan can run efficiently without relying on restricted technology.
Large language models are gaining momentum in China, with investments from companies like Baidu, Zhipu.ai, and IDEA (led by Harry Shum). This growth aligns with China’s preparations to enforce stringent AI regulations, particularly in the realm of generative AI content. The country’s focus on content control may require companies to obtain licenses before launching large language models, potentially impacting China’s competitiveness in the burgeoning industry compared to the U.S.
Baichuan’s quick progress over the past three months is evident with a team of 50 members by April and the successive releases of Baichuan-7B and Baichuan-13B. The foundational model, Baichuan-13B, is currently available for free to academics and developers with official permission for commercial use.
Looking ahead, Wang Xiaochuan stated in an interview that his startup intends to release a large-scale model similar to OpenAI’s GPT-3.5 by the end of the year. With Baichuan’s advancements and China’s growing focus on AI regulation and development, the country is poised to compete with the U.S. in the field of large language models.
