Papers

Filtered to hybrid LLMs · clear filter

Browse by term

continual learning 83reinforcement learning 50large language models 12benchmarking 11benchmarks 11language models 11vision-language models 11robotics 7natural language processing 6world models 6generative models 5recursive self-improvement 5attention mechanisms 4diffusion Transformers 4multi-agent systems 4multimodal learning 4multimodal models 4on-policy distillation 4self-distillation 4self-supervised learning 4transformers 4video generation 4vision-language-action models 4agent-based systems 3agentic models 3agentic search 3autonomous systems 3coding agents 3diffusion models 3image generation 3

Matching papers

Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM

73 upvotes · 3 SEP 2026 · Sergii Kozyrev, Davyd Maiboroda

This paper investigates why a specific neural network architecture, Gated DeltaNet, can survive 4-bit quantization. The researchers found that a new quantization technique, NVFP4, can be used to quantize the recurrent state of GDN without significant loss of performance, and they provide a mechanistic explanation for why this is the case.

continual learninghybrid LLMsquantizationattentionrecurrent neural networks