The Allen Institute for AI (AI2) has announced the development of AI2 OLMo, an open language model designed for scientific research. The model, being developed in collaboration with AMD and CSC, will be powered by the LUMI supercomputer. OLMo will be comparable to other large language models with 70 billion parameters and is expected to launch in 2024. The model will be open-source, providing access to all aspects of its creation. Key individuals involved include Hannaneh Hajishirzi and Noah Smith, Senior Directors of NLP Research at AI2, Ian Ferreria, Senior Director of AI Solutions at AMD, and Pekka Manninen, Director of Science and Technology at CSC.
AI2 Announces the Development of OLMo, an Open Language Model
The Allen Institute for AI (AI2) has recently announced the development of a new open language model, AI2 OLMo (Open Language Model). This model, which is expected to be launched in early 2024, will be on par with other large language models in terms of scale, boasting 70 billion parameters. The development of OLMo is a collaborative effort between AI2, AMD, and CSC, utilizing the GPU portion of the LUMI pre-exascale supercomputer, which is powered by an all-AMD processor. LUMI is recognized as one of the most environmentally friendly supercomputers in the world.
OLMo: A Unique Open Language Model for the Research Community
OLMo is designed to be a unique open language model that will provide the research community with access and education around all aspects of model creation. This model will offer a new opportunity for many individuals in the AI research community to work directly on language models for the first time. All elements of the OLMo project will be accessible, including the data, the code used to create the data, the model, the training code, the training curves, and evaluation benchmarks. The ethical and educational considerations surrounding the creation of this model will also be openly shared and discussed to guide the understanding and responsible development of language modeling technology.
OLMo: A Tool for Advancing the Science of Language Models
OLMo is being developed with the scientific community in mind and is intended to advance the science of language models. According to Hannaneh Hajishirzi, an OLMo project lead and a Senior Director of NLP Research at AI2, OLMo will be the first language model specifically designed for scientific understanding and discovery. The collaboration between AI2’s expertise in natural language processing (NLP) and AMD’s history of supporting the scientific community through high-performance computing efforts makes this project a perfect match.
OLMo: A Collaboration between AI2, AMD, and LUMI
The OLMo initiative is a collaborative effort between AI2, AMD, and LUMI. According to Ian Ferreria, senior director, AI Solutions, AMD, this initiative will extend their knowledge into generative AI using the impressive capabilities from the LUMI Supercomputer powered by AMD EPYC™ CPUs and AMD Instinct™ accelerators. Noah Smith, an OLMo project lead and a Senior Director of NLP Research at AI2, also notes that AI2 has the unique ability to bring their world-class expertise together with world-class hardware from AMD and LUMI to produce something explicitly designed for scientists and researchers.
OLMo: A Potential Breakthrough Technology
Generative AI, such as OLMo, carries the potential of being the breakthrough technology of this decade, similar to how search engines and smartphones revolutionized society in previous decades. Pekka Manninen, Director of Science and Technology at CSC, emphasizes the importance of open, transparent, and explainable Large Language Models (LLMs) for the democratization of this technology. He also highlights the role of supercomputers like LUMI in accelerating LLM training and positioning it as a platform for natural language processing.
