Meta, the parent company of Facebook, has just announced the release of Llama 2, a new family of AI models aimed at improving text generation for apps like OpenAI’s ChatGPT and Bing Chat. This new release follows the previous Llama model, which was only available by request due to concerns about misuse. However, Llama later leaked online and spread across various AI communities.
Unlike its predecessor, Llama 2 will be accessible for fine-tuning on platforms such as AWS, Azure, and Hugging Face’s AI model hosting platform. It has also been optimized for Windows and Qualcomm Snapdragon devices. In fact, Qualcomm aims to bring Llama 2 to Snapdragon devices by 2024.
So, what sets Llama 2 apart from Llama? According to Meta’s whitepaper, Llama 2 comes in two versions: Llama 2 and Llama 2-Chat, with the latter being fine-tuned specifically for two-way conversations. Furthermore, both versions are divided into different levels of sophistication based on the number of parameters they possess. Meta claims that Llama 2’s performance has significantly improved compared to its predecessor.
Llama 2 was trained using two million tokens, which is almost double the amount used for training Llama. Generally, more tokens lead to better performance in generative AI models. For reference, Google’s current large language model (LLM), PaLM 2, was trained on 3.6 million tokens, and GPT-4 is speculated to have been trained on trillions of tokens.
Although Meta hasn’t disclosed the specific sources of the training data, they’ve stated that it comprises publicly available web text in English, particularly of a factual nature. Nevertheless, Meta acknowledges that biases still exist within Llama 2 models, such as a tendency to generate more he pronouns than she pronouns due to imbalances in the training data. Additionally, Llama 2 has a Western skew, likely caused by an overabundance of terms related to Christianity, Catholicism, and Judaism.
Despite Llama 2’s slight performance lag behind closed-source rivals like GPT-4 and PaLM 2, Meta claims that human evaluators find Llama 2 just as helpful as OpenAI’s ChatGPT, with both models performing equally across 4,000 prompts designed to measure helpfulness and safety.
While Meta has taken precautions to minimize potentially harmful outcomes, such as leveraging Microsoft’s Azure AI Content Safety service to reduce toxic outputs, they emphasize that users must comply with Meta’s license and guidelines for safe development and deployment.
By openly sharing Llama 2, Meta hopes to foster the development of more helpful and safer generative AI. However, the concern remains about how and where these models will be used, as open source models can be adopted rapidly and widely on the internet.
In conclusion, Meta’s release of Llama 2 aims to enhance text generation capabilities for various chatbot applications. Despite some limitations and biases, Llama 2 shows promise in terms of helpfulness and safety. As the AI landscape continues to expand, it will be crucial to address concerns regarding bias and ethical use to ensure these models benefit society at large.
