Firehose

Filtered to Papers, tagged “mixture-of-experts” · clear filters

Browse: People · Companies · Papers · Podcasts · Hacker News · Deep dives

All People Companies Papers Podcasts Hacker News

Browse by tag

continual learning 26 reinforcement learning 9 language models 5 large language models 4 robotics 4 agents 3 generative models 3 multi-agent systems 3 recursive self-improvement 3 vision-language models 3

16 SEP 2026 · Paper

The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction

This paper presents a new technique to reduce memory usage and speed up inference for large neural networks, allowing them to run on consumer hardware with limited memory. A practitioner might care about this because it enables the deployment of large models in edge devices and reduces the need for expensive storage.

mixture-of-experts memory efficiency deep learning inference neural network pruning sparse models

13 SEP 2026 · Paper

Flattening Every Memory Peak in Long-Context Mixture-of-Experts Training

This paper addresses the issue of memory peak allocation in Mixture-of-Experts (MoE) models during long-context training, proposing four different techniques to reduce memory usage without compromising performance. Practitioners might care about these techniques to train larger MoE models with longer context lengths.

mixture-of-experts memory-efficient training long-context models GPU optimization model parallelism activation management