The huge explosion of generative AI models for text and images has recently become unavoidable. As these models become more and more capable, “base model” is a relatively new term in use. So what is a foundation model?
The term remains somewhat vague. Some define it by the number of parameters, and thus how big a neural network is, and others by the number of unique and difficult tasks the model can perform. Is the ever-expanding size of AI models and the model’s ability to tackle multiple tasks really that exciting? Stripping away all the hype and marketing talk, what’s really exciting about these new generations of AI models is this: they’ve fundamentally changed the way we interact with computers and data. Think of companies like Cohere, Covariant, Hebbia and You.com.
We have now entered a critical phase of AI where who gets to build and operate these powerful models has become a major talking point, especially as ethical issues begin to swirl, such as who is entitled to what data, or whether models violate reasonable assumptions of privacy , whether data use consent is a factor, what constitutes “inappropriate conduct” and much more. With questions like these on the table, it’s reasonable to assume that those in control of AI models may well be the most important decision-makers of our time.
Is there a game for open source basic models?
Due to the ethical issues associated with AI, the call for open source basic models is gaining momentum. But building foundation models is not cheap. They require tens of thousands of state-of-the-art GPUs and many machine learning engineers and scientists. The realm of building foundation models has so far only been accessible to the cloud giants and extremely well-funded startups sitting on hundreds of millions of dollars war chest.
Almost all of the models and services built by these handpicked companies are closed source. Yet closed source entrusts an awful lot of power and decisions to a limited number of companies that will determine our future, which can be quite disturbing.
We have entered a critical phase of AI where who gets to build and operate these powerful models has become a major point of discussion.
However, Stability AI’s open sourcing of Stable Diffusion posed a serious threat to the builders of the base models who were determined to keep all the secret sauce to themselves. Applauds have been heard from developer communities around the world for Stability’s open sourcing as it liberates systems and puts control in the hands of the masses versus select corporations who could be more interested in profit than what’s good for humanity. This is now influencing the way insiders think about the current paradigm of closed source AI systems.
Potential obstacles
The biggest obstacle to open sourcing base models remains money. Making open source AI systems profitable and sustainable still requires tens of millions of dollars to be managed and managed properly. While this is a fraction of what the big companies invest in their efforts, it is still quite significant for a startup.
We can see that Stability AI’s attempt to open source Neo-GPT and turn it into a real business failed, as it was outclassed by companies like Open AI and Cohere. The company must now idealize with the Getty Images lawsuit, which threatens to distract the company and further deplete resources – both financial and human. Meta’s opposition to closed source systems via LLaMA has poured gas into the open source movement, but it’s too early to tell if they’ll continue to deliver on their promise.
