The gap between open-source AI initiatives and major private companies goes beyond access to computing power. Ai2, formerly the Allen Institute for AI, aims to bridge this divide with fully open databases, models, and a new post-training framework called Tülu 3, designed to make large language models (LLMs) more functional.
While pretraining is essential for creating foundation models, it’s not enough to make them useful. Post-training is where raw LLMs are refined, shaping them from general-purpose networks into specialized tools. This process helps eliminate issues such as inappropriate or irrelevant outputs while tailoring the model to specific use cases, such as research or therapy.
Unlike private companies that keep their post-training methods proprietary, Ai2 is committed to transparency. Although models like Meta’s Llama are described as open-source, the methods used for their refinement are kept confidential. Ai2’s approach stands out by openly sharing its data collection, curation, and training methods, as seen in its work on models like OLMo.
However, conducting post-training remains challenging for most developers. It requires technical expertise, significant time, and often reliance on external companies or tools, which can be costly and introduce risks, particularly for sensitive fields like healthcare.
Ai2’s Tülu 3 aims to change this by democratizing post-training. This updated system builds on the earlier Tülu 2 framework, offering significant improvements in model customization and performance. Through processes like data curation, reinforcement learning, fine-tuning, and preference tuning, Tülu 3 enables users to tailor models to prioritize specific skills, such as math or coding, while de-emphasizing others.
The broader goal is to reduce dependence on private companies for creating custom-trained models. Organizations, especially those handling sensitive data, can now avoid external APIs or third-party services, opting instead for an on-premises solution powered by Tülu 3.
Ai2 has already applied Tülu 3 to Llama-based models and plans to release an OLMo-based model soon. This forthcoming version will be fully open-source, reflecting Ai2’s mission to make AI development more accessible.
For those curious about Tülu 3’s performance, Ai2 offers a live demo to explore its capabilities.
