Generative AI has been one of the most sought-after technologies of recent years, with Google and OpenAI among the key players in this market. But, according to a leaked memo from a Google AI engineer, these companies may not be well-positioned to dominate the generative AI market. Instead, the open-source community could be the hidden third party that could eat Google and OpenAI’s AI lunch.
How is this possible? The key lies in a recent leak of the Meta AI Model (LLaMA), which has sparked an avalanche of innovation from the open-source community. Despite the model’s initial lack of instruction and dialogue tuning, it was quickly iterated upon, with enhancements like instruction tuning, quantization, and quality improvements being developed in quick succession. One game-changer is a cheap fine-tuning mechanism called Low-Rank Adaptation (LoRA), which allows for fine-tuning of the model at a fraction of the cost and time.
This technology has greatly reduced the barriers to entry for training and experimentation, allowing individuals to personalize language models on their hardware in a matter of hours. The open-source community has also efficiently used high-quality, curated datasets for training, following the philosophy that data quality is better than data size. These datasets are typically developed using synthetic techniques and collected from other projects.
As a result, recent advances by the open-source community have prompted a reassessment of Google and OpenAI’s strategy. Open-source AI models are an attractive alternative for many users due to their rapid innovation and unlimited use. It also means that generative AI is within the reach of almost any AI-savvy open-source developer.
It’s worth noting that FAANG (Facebook, Apple, Amazon, Netflix, Google) has benefited so far from generative AI, but all their work has been based on open-source AI programs. Without TensorFlow, PyTorch, and Hugging Face’s Transformer, there would be no ChatGPT or Bard.
So, it seems like betting on open source is the way forward for generative AI. As the mystery developer at Google put it, Competing directly with open source is a losing proposition. There’s a reason: Open source has some key advantages that we can’t replicate.
