Join our newsletters for the latest updates and exclusive content on AI advancements.
Silo AI, a startup based in Helsinki, Finland, has introduced Poro, a new open-source large language model (LLM) aimed at boosting multilingual AI capabilities across European languages. Poro is the first in a series of models planned to cover all 24 official European Union languages. These models are being developed by SiloGen, Silo AI’s generative AI division, and the University of Turku’s TurkuNLP research group.
Peter Sarlin, CEO of Silo AI, highlighted the importance of digital sovereignty, ensuring that models capture European values, culture, and languages. He emphasized that this allows companies to create proprietary models that retain value within Europe.
Poro 34B, named after the Finnish word for “reindeer,” has 34.2 billion parameters and uses a BLOOM transformer architecture with ALiBi embeddings. It was trained on a dataset that includes English, Finnish, and programming languages like Python and Java. The training took place on LUMI, Europe’s fastest supercomputer, located in Kajaani, Finland.
Sarlin explained that Poro aims to address the challenge of training effective natural language models for less-resourced European languages like Finnish. The model leverages cross-lingual training, using data from higher-resourced languages like English to improve its performance.
Poro is the second major open-source LLM from Europe, following the French startup Mistral AI’s Mistral 7B. This development underscores Europe’s growing achievements in the generative AI field and highlights the competitive landscape among AI labs and companies.
As part of their transparency commitment, SiloGen will document Poro’s training progress through the Poro Research Checkpoints program. This includes releasing training checkpoints, a novel approach that provides transparency. Initial benchmarks show that Poro, even at 30% training completion, surpasses existing monolingual Finnish AI models like FinGPT on the FIN-bench evaluation.
Despite its multilingual capabilities, Poro also performs well in English, matching or exceeding existing models on standard English evaluation sets. This demonstrates that multilingual proficiency does not detract from its performance in more widely-used languages.
Sarlin advocates for open-source models like Poro as they offer a transparent and ethical alternative to closed models from major tech companies. He believes that the future of AI lies in open source, providing full visibility into model development and ensuring regulatory compliance by design.
Silo AI plans to continue releasing Poro checkpoints during its training. The ultimate goal is to create a family of open-source models covering all European languages, challenging Big Tech’s dominance.
Poro reflects the ongoing partnership between Silo AI and the University of Turku, leveraging the University’s expertise in multilingual language modeling. Sarlin noted that the collaboration combines Silo AI’s applied AI expertise with the University’s research capabilities, particularly benefiting lower-resourced European languages.
The release of Poro marks a new era of open collaboration and transparency in natural language processing. Initiatives like Poro Research Checkpoints aim to democratize access to advanced tools and insights previously controlled by tech giants.
Many large enterprises, including Allianz, Rolls Royce, Honda, and Philips, are already concerned about future regulations and which AI models they can use. Poro’s success could democratize access to effective multilingual models, offering Europe an alternative to US-developed systems. Though it’s early days, Poro’s release is a significant step towards open, transparent language AI.