Today, Meta AI released LIMA, a 65 billion parameter language model trained with the standard supervised loss on only 1,000 prompts and responses. This new language model has demonstrated strong capabilities in learning specific response formats and handling complex queries without needing to rely on reinsurance learning or human preference modelling. Interestingly, LIMA has also been seen to generalize well when applied to unseen tasks despite its limited training data. When compared to existing language models such as GPT-4, Bard, and DaVinci-003, LIMA has been preferred in 43%, 58%, and 65% of cases, respectively.
Meta AI, along with Carnegie Mellon University, University of Southern California, and Tel Aviv University, have worked together to develop this new breakthrough language model. The team has recently released MEGABYTE, a scalable architecture for modelling long sequences, which outperformed existing byte-level models across a range of tasks and modalities. This new technology is designed to handle large models of sequences of over 1 million tokens.
When it comes to the Meta AI team, it is made up of experienced professionals who have extensive experience in artificial intelligence and machine learning. They are headed by Graeme, their CEO, who has worked in the AI field for over 15 years. Graeme previously founded another AI startup, Minden Road, and has years of experience in understanding and applying AI technology. He has been recognised for his work, and he strongly believes in the value that AI can bring to businesses and society.
