Apple Surreptitiously Introduces an Open Source Multimodal LLM in October

Apple Surreptitiously Introduces an Open Source Multimodal LLM in October

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More

In October 2023, researchers from Apple and Columbia University quietly launched Ferret, an open-source multimodal large language model (LLM). Initially, this release, which included code and weights for research purposes only, didn’t attract much attention. However, discussions around local LLMs powering small devices have intensified, particularly with recent developments from companies like Mistral and Google.

Apple recently announced a significant breakthrough in deploying LLMs on iPhones. They published two research papers detailing new techniques for 3D avatars and efficient language model inference. These advancements could lead to more immersive visual experiences and enable complex AI systems to run on consumer devices such as iPhones and iPads.

The AI community has started to recognize Apple’s unexpected move into the open-source LLM arena. Traditionally known for its closed ecosystem, Apple’s involvement in open-source AI has surprised many. Bart de Witte, head of a European non-profit focused on open-source AI in medicine, noted his excitement on social media, praising Apple’s contribution to impactful AI research.

Similarly, German AI music artist and advisor Tristan Behrens highlighted the release of Apple’s multimodal LLM on social media, noting the significance of including both code and weights. Tech blogger and VentureBeat contributor Ben Dickson expressed his surprise at Apple’s entry into the open-source LLM field, given the company’s history of being a closed system. He reasoned that Apple’s move makes sense in the context of competing with models like ChatGPT, as Apple’s infrastructure isn’t designed to support LLMs at scale independently.

This news comes at a time when both Anthropic and OpenAI are reportedly seeking significant funding for their proprietary LLM developments. Reuters mentioned that Anthropic is in talks to raise $750 million, while Bloomberg reported that OpenAI is looking to secure new funding at a valuation of $100 billion or more.