Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage.
VentureBeat and other experts have argued that open-source large language models (LLMs) might significantly influence generative AI in enterprises more than closed models, like OpenAI’s ChatGPT or Anthropic’s competitors. However, proving this has been challenging due to the limited number of actual deployments. While there is extensive experimentation with open-source models, few established companies have publicly announced their use in real business applications.
To explore this further, we contacted major open-source LLM providers such as Meta and Mistral AI, and distributors like IBM, Hugging Face, Dell, Databricks, AWS, and Microsoft. From our interviews, it appears that several initial public examples exist (we found 16 notable cases), but it’s still early days. Industry observers expect the number of cases to rise significantly later this year.
One reason for the delay is that open source was initially slower to launch. Meta released the first major open-source model, Llama, in February 2023, three months after OpenAI’s ChatGPT came out in November 2022. Mistral AI released Mixtral, a top-performing open-source LLM, in December 2023. Consequently, deployment examples are just starting to emerge. Advocates of open source believe it’s only a matter of time before their models catch up with closed-source models.
There are some limitations with current open-source models. For instance, Amjad Masad, CEO of Replit, pointed out that the feedback loop isn’t working properly because contributing to model development isn’t easy. People underestimated the extent of experimentation that would occur with open-source models. Developers have created thousands of derivatives of models like Llama, and they are steadily achieving parity or even outperforming closed models in certain metrics, such as FinGPT, BioBert, Defog SQLCoder, and Phind.
Matt Baker, SVP of AI Strategy at Dell, which partners with Meta to bring Llama 2 to enterprise users, argues that large public models offer little value to private companies. Enterprises need models that can access their own data easily, and most of the AI work involves integrating these models with company-specific data through techniques like retrieval augmented generation (RAG), which isn’t always reliable.
Many enterprises are building open-source-based applications for customer support and code generation. These applications can interact with custom code that might not be understandable to general closed-model LLMs like those from OpenAI or Anthropic, which prioritize popular cloud languages over legacy enterprise code.
Hugging Face, a major provider of open-source LLM infrastructure, supports hundreds of thousands of developers. According to Andrew Jardine from Hugging Face, enterprise companies are cautious about moving forward with LLM applications due to considerations around data privacy, customer experience, and ethics. Companies usually start with internal use cases to validate the technology before deploying it externally.
Despite the hesitation, some believe that relying solely on open-source models can be too much work. It’s easier to use an API from OpenAI, which provides on-demand cloud services and indemnification, thus avoiding licensing and governance challenges associated with open source. Furthermore, GPT models generally perform well across multiple languages, whereas open-source LLMs can be inconsistent.
However, the distinction between open and closed models is increasingly blurred. Most enterprises are likely to use both. Jardine cites a pharmaceutical company that uses a closed LLM for internal chatbots while leveraging Llama for detecting sensitive information, appreciating the control over data that open source provides. Concerns about data control and model updates also drive companies to adopt open-source solutions.
We found several companies, such as Intuit and Perplexity, employing multiple models within a single application. This approach allows them to select the best LLM for specific tasks, using orchestration layers to call the most appropriate model for each use case. While deploying open-source models can be cumbersome initially, it can become more cost-effective in the long term, especially for companies with their own infrastructure, as they avoid paying for proprietary IP and development costs.
Some companies are quietly deploying open-source models to maintain control and achieve customization. For instance, Intuit uses a mix of internal and open-source models for its Intuit Assist feature. Walmart has developed numerous conversational AI applications, mixing open-source models like Google’s BERT with others to avoid being locked into a single provider’s ecosystem.
Additionally, companies like IBM are leveraging open-source LLMs for various applications, including internal HR support and external services like AI-based sports commentary. Despite the challenges and considerations involved, the adoption of open-source LLMs in enterprise environments is growing, driven by the need for control, customization, and cost-efficiency.
Below are examples of enterprise companies actively using open-source LLMs:
1. VMWare: Using HuggingFace’s StarCoder model to help developers generate code efficiently.
2. Brave: Deploying Mixtral 8x7B from Mistral AI in its Leo conversational assistant.
3. Gab Wireless: Leveraging Hugging Face models to screen messages for children’s safety.
4. Wells Fargo: Utilizes Meta’s Llama 2 model for internal applications.
5. IBM: Integrating open-source LLMs into various tools for HR, consulting, and marketing.
6. Grammy Awards: Using Llama 2 for generating custom AI content.
7. Masters Tournament, Wimbledon, and US Open: Employing open-source LLMs for sports commentary.
8. Perplexity: Deploying custom-built open-source models for search experiences.
9. CyberAgent: Using Dell software to power OpenCALM, a Japanese language model.
10. Intuit: Mixing internal and open-source models for its Intuit Assist feature.
11. Walmart: Combining various models for customer care and other applications.
12. Shopify: Utilizing Llama 2 in its AI-powered tool Sidekick.
13. LyRise: Using Llama for its talent-matching chatbot.
14. Niantic: Integrating Llama 2 into its game for environment-specific character reactions.
These examples highlight how open-source LLMs are being adopted across different industries, providing enterprises with greater control and customization while addressing specific business needs. The trend indicates a growing confidence in the capabilities of open-source models to meet enterprise demands.