Nathan Lambert
AI alignment & RLHF
AI researcher, formerly at Hugging Face. Writes the Interconnects newsletter on AI alignment and RLHF.
Recent activity
-
Here is a summary of the article in 3 plain sentences for a developer/AI-ML audience: Open-source AI models have gained significant attention in recent years, with leading models coming from Chinese labs, and their adoption is expected to continue growing, with the open-closed model gap narrowing to around 4-6 months. However, concerns around safety and cybersecurity risks associated with open models, such as distillation, remain, with some arguing that distillation is a key factor behind Chinese models' performance, while others argue it is overstated. The US government is taking notice of the trend, with regulatory attention focused on companies using Chinese models, and experts advocating for a national AI cybersecurity policy to address emerging threats. AI summary
Read more → -
The resignation of Jacob Coxon, citing safety risks in AI, has sparked a wildfire of fear and discussion about existential risk and mass extinction, despite the lack of concrete evidence. The article argues that the discourse around existential risk is on poor footing, with many researchers overstating the risks and downplaying the benefits of AI advancements. The author suggests that the real risks are more focused on AI-caused disasters, such as cyber attacks or bio-risks, which are worth debating but not necessarily worth panicking about. AI summary
Read more → -
The average person is unlikely to feel the impact of AI in their daily life for at least 5 years, and possibly not until the 50-year mark, as the benefits of AI will be indirect and not immediately tangible. The AI industry is facing a challenge in building foundational infrastructure and a general process that will compound over decades, with the most important part of what's happening early in the AI revolution being the creation of a general process that will lead to significant advancements in the future. The benefits of AI will not be evenly distributed, with the majority of society not benefiting from the technology, leading to a potential backlash against the industry. AI summary
Read more → -
Several new open models have been released, including Motif-3 with an MIT license, GLM-5.3 with a custom license requiring security reviews for commercial use, and Hy4-preview by Tencent, which has shown promise despite an initial issue with overthinking. Additionally, models like dots3-note-prev and Qwen3.8-Flash-Next have demonstrated improvements in architecture and performance. AI summary
Read more → -
Nvidia aims to democratize AI model development by releasing open-source models with training data, code, and weights, encouraging users to build their own token machines, thereby reducing monopolization of intelligence. However, this approach faces challenges due to the capital-intensive nature of training models, with Nvidia reportedly spending $26 billion on this endeavor. The future of open-source AI depends on whether this model can become self-sustaining, with two possible outcomes: either it succeeds and becomes a viable alternative to closed models, or it forks into a different development path. AI summary
Read more → -
Z.ai's GLM-5.3 model has surpassed Moonshot AI's Kimi K3 and some of the top models like Claude Fable 5 or GPT-5.6-Sol in agentic coding benchmarks, thanks to a robust post-training regime that involves "more environments, more diverse tasks, and more compute spent training on them." This approach allows the Chinese lab to stay competitive with leading American models, despite having fewer parameters. AI summary
Read more → -
Researchers and developers may be disappointed to learn that current large language models (LLMs) struggle to produce high-quality, long-form non-fiction writing, despite their capabilities in other areas like coding and mathematics. AI summary
Read more → -
The book "Reinforcement Learning from Human Feedback: Aligning and Post-training LLMs" covers three key areas: 1. Intuitions for how RL algorithms change model outputs, including explanations of policy-gradient theorem, PPO, GSPO, and CISPO. 2. Understanding the crucial factors facing new RL systems and algorithms, such as off-policy data, training-inference mismatch, and throughput. 3. The histories that led to modern post-training, including the evolution of RL from preferences to language models and the impact of ChatGPT. AI summary
Read more → -
The recent hacks on in-development frontier models highlight the need for a more balanced approach to model development, with both companies and governments prioritizing transparency and risk mitigation. Frontier labs' fast-paced development and lack of oversight contribute to the risk of misaligned models, while the government's slow response to emerging AI risks is a pressing concern. Open models are essential for advancing public understanding of frontier AI risks, but their proliferation must be carefully managed to prevent the spread of malicious capabilities. AI summary
Read more → -
Interconnects has launched an Artifacts Hub and Adoption Dashboard to provide free, curated data on the open model ecosystem, including inference tokens, model intelligence, and adoption metrics. The Adoption Dashboard offers insights into download and derivative model numbers by geography and organization, highlighting the US-China gap and growing players. The data is available for free to support the growth of the open ecosystem. AI summary
Read more → -
Several open models have been released, showcasing their utility on the Pareto frontier, including Inkling by Thinking Machines, Hy3 by Tencent, Laguna S2.1 by Poolside, and Kimi K3 by Moonshot AI, which demonstrate improvements in performance and efficiency. These models are pushing the boundaries of what is possible with open models, with some companies like Thinking Machines and Tencent generating hundreds of millions in revenue per year from their open model finetuning services. The increasing adoption of open models is expected to continue, with potential implications for the AI industry. AI summary
Read more → -
Nathan and Florian discuss the recent release of Kimi K3, a Chinese open model, and its performance gap to the US-based models, with some benchmarks suggesting it's 2-6 months behind. They also touch on Qwen's announcement of an open-weight model, Xi's commitment to openness and open-source, and the growing trend of distillation and fine-tuning in the open model ecosystem. AI summary
Read more → -
Moonshot AI's release of Kimi K3, a 2.8T parameter model, has narrowed the performance gap between open and closed models from 6-9 months to 3-5 months, with Kimi K3 being the strongest open model ever released, rivaling the performance of closed models from Anthropic, OpenAI, and DeepMind. The model's success suggests that Chinese companies are capable of building high-quality models without relying on IP theft, and Moonshot AI's approach is more extreme than previously thought. The open weights release of Kimi K3 will likely have significant implications for the AI ecosystem, including increased competition and a need for coordination as powerful technologies are rolled out globally. AI summary
Read more → -
The most serious test to date of open source AI’s viability is happening right now.
Read more → -
Latest open artifacts (#22): Zyphra, Cohere, and Poolside are expanding the breadth of the ecosystem
An assessment of the open ecosystem and the motivations behind releasing models
Read more → -
Z.ai's GLM-5.2 model has crossed a significant user experience threshold, offering a step change in performance and capabilities, particularly in open agent applications, and has gained widespread praise from the AI community. The model's release has accelerated the development of open-weight models, with GLM-5.2 being the first to offer credible alternatives to Anthropic's closed models, and is poised to drive further adoption and innovation in the field. This marks a significant milestone in the evolution of open-source AI models. AI summary
Read more → -
Banning open source AI would be a grave mistake as it drives economic growth, promotes education, innovation, and competition, and is inherently secure and transparent. Over 90% of the world's software is built on open source, producing over $8 trillion in economic benefits, and open source AI is quietly improving and securing AI models everywhere. The US government's recent actions to regulate AI could inadvertently or intentionally ban open source, which would have unintended consequences, particularly for startups and educational institutions that rely on it. AI summary
Read more → -
About 3 years since I started writing weekly.
Read more → -
Finbarr Timbers discusses the evolution of post-training recipes in large language models, highlighting the shift from a single pipeline to multi-stage recipes with the emergence of Multi-teacher On-Policy Distillation (MOPD) in 2026 models such as MiMo Flash V2 and Nemotron 3 Ultra. MOPD involves training multiple domain-specialist teachers and distilling their knowledge into a single student using on-policy distillation. This approach enables scalability and flexibility in post-training, addressing the limitations of traditional methods. AI summary
Read more → -
It's a one-way door and we weren't ready for it.
Read more →