Join our newsletters for daily and weekly updates and exclusive content on the latest in AI.
About a year ago, and just two weeks before ChatGPT was released by OpenAI, Meta introduced a research tool called Galactica. This open-source “large language model for science” was trained on a vast dataset that included 48 million scientific papers. Meta promoted Galactica’s ability to summarize academic literature, solve math problems, create Wiki articles, write scientific code, annotate molecules and proteins, and much more.
However, Galactica was publicly available for merely three days. On November 17, 2022, Meta removed the demo after widespread criticism, particularly about what was then an emerging issue: hallucinations. Users were shocked by the model’s occasionally inaccurate and even offensive outputs, which were presented in a way that seemed credible. Despite a series of defensive tweets from Meta’s head scientist, Yann LeCun, arguing in favor of the model, Galactica did not become the groundbreaking tool it was hoped to be.
Two weeks later, ChatGPT was launched. Around the same time, there were tantalizing rumors about the imminent release of GPT-4. On November 30, amidst hopeful whispers at the NeurIPS conference in New Orleans, OpenAI released ChatGPT.
Like Galactica, ChatGPT also faced issues with hallucinations, providing responses that seemed convincing but were sometimes inaccurate. OpenAI addressed this flaw openly, acknowledging the difficulty in resolving it. Despite this, ChatGPT’s popularity skyrocketed. It became one of the fastest-growing services ever, reaching 100 million monthly users within two months and now boasting 100 million weekly users.
Galactica’s impact remains significant for Meta. Joelle Pineau, VP of AI research at Meta, mentioned that many valuable lessons were learned from Galactica’s rollout. Pineau clarified that Galactica was always intended as a research project and not a finished product. It was released quietly on GitHub, but the excitement around it led to expectations beyond what was achievable with scientific research alone. The gap between public expectations and the model’s capabilities was wide. While some of Galactica’s hallucinations were less severe than in other models, people still expected it to perform flawlessly, which was never the intent.
The experiences from Galactica influenced the release strategy for Meta’s subsequent model, Llama. Pineau explained that Meta took down Galactica to prevent misuse and emphasized the importance of including guidelines for responsible usage. The setbacks with Galactica affected how they managed Llama’s release, ensuring more controlled expectations and better handling of public reaction.
Released on February 24, 2023, Llama marked a significant step with Meta committing to more open research. The release was followed by Llama 2 in July and Code Llama in August. Despite some debates about the full openness of these models, Llama has been a pivotal point in the AI landscape. Yann LeCun pointed out that researchers had to request access to Llama as a lesson learned from the Galactica experience, where the public reaction had been overwhelmingly negative.
In summary, the lessons from Galactica were crucial in shaping Meta’s approach to future AI models, ensuring better preparation and management of public expectations.