Firehose

Filtered to tagged “language models” · clear filters

Browse: People · Companies · Papers · Podcasts · Hacker News · Deep dives

All People Companies Papers Podcasts Hacker News

Browse by tag

30 JUL 2026 · Paper

Memory Decoder at Scale: A Pretrained, Parametric Long-Term Memory

This paper develops a method to scale up language models by increasing their memory capacity, allowing for better performance and more efficient use of parameters. Practitioners may care about this research as it could lead to more powerful and efficient language models for applications like language translation and text generation.

long-term memory language models memory capacity parameter scaling continual learning

30 JUL 2026 · Paper

Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations

This paper develops a technique to identify and mitigate demographic bias in large language models by selectively pruning specific neurons in neural networks, without significantly impacting the model's overall performance. Practitioners in AI and NLP may care about this method as it could help create more fair and transparent language models.

fairness bias neural pruning language models deep learning

30 JUL 2026 · Paper

Beyond Geometric Complementarity: Coherent Overlap in Sparse Mixture-of-Experts Routing

This paper investigates how sparse mixture-of-experts language models route tokens to multiple experts and how this routing affects their performance. Practitioners may care because understanding how to optimize these models can lead to better language understanding and generation.

sparse mixture-of-experts routing representation overlap redundancy language models

30 JUL 2026 · Paper

β-OPSD: Deriving with Policy Optimization, Training with Self-Distillation

This paper develops a new method for improving reasoning language models, called β-OPSD, which combines policy optimization and self-distillation to improve stability and performance. Practitioners might care about this method because it provides a more efficient and effective way to improve language model reasoning abilities.

policy optimization self-distillation continual learning language models reinforcement learning

29 JUL 2026 · Paper

Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modes

This paper studies how lossy verification schemes can improve the efficiency of speculative decoding in large language models, but may also degrade generation quality. Practitioners may care about understanding the trade-offs between speed and quality when using these schemes.

continual learning language models speculative decoding verification acceleration

29 JUL 2026 · Paper

MindForge: Teaching Small Language Models Whole-Life-Cycle Software Engineering via Source-Free Program Synthesis

This paper teaches language models to synthesize complete software programs from scratch, which is a challenging task. Practitioners might care because this can improve the models' performance on software engineering tasks.

program synthesis software engineering language models continual learning benchmarks

1 JUL 2026 · Podcast · Machine Learning Street Talk (MLST)

The Benchmark With No Instructions — ARC-AGI-3 (winning team!)

Tim Scarfe interviews the Tufa Labs ARC-AGI-3 team to dissect their winning approach on the ARC-AGI-3 benchmark, focusing on how their system discovers goals and balances exploration with action efficiency. The episode explores the challeng…

ARC-AGI-3 benchmark goal acquisition action efficiency exploration vs exploitation language models planning in AI coding agents requirements engineering core knowledge priors abstraction synthesis reinforcement learning AI safety bitter lesson neural guided search LLM reasoning software engineering AI benchmark design general intelligence