Papers

Filtered to attention · clear filter

Browse by term

continual learning 83reinforcement learning 50large language models 12benchmarking 11benchmarks 11language models 11vision-language models 11robotics 7natural language processing 6world models 6generative models 5recursive self-improvement 5attention mechanisms 4diffusion Transformers 4multi-agent systems 4multimodal learning 4multimodal models 4on-policy distillation 4self-distillation 4self-supervised learning 4transformers 4video generation 4vision-language-action models 4agent-based systems 3agentic models 3agentic search 3autonomous systems 3coding agents 3diffusion models 3image generation 3

Matching papers

Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM

73 upvotes · 3 SEP 2026 · Sergii Kozyrev, Davyd Maiboroda

This paper investigates why a specific neural network architecture, Gated DeltaNet, can survive 4-bit quantization. The researchers found that a new quantization technique, NVFP4, can be used to quantize the recurrent state of GDN without significant loss of performance, and they provide a mechanistic explanation for why this is the case.

continual learninghybrid LLMsquantizationattentionrecurrent neural networks

Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction

64 upvotes · 8 SEP 2026 · Vishesh Tripathi, Abhay Kumar, Ramsha Khan

This paper introduces Grouped Value Attention (GVA), a technique to reduce the memory footprint of Transformer decoding by storing grouped values and reconstructing content keys. Practitioners might care about this because it could lead to faster and more efficient models.

Transformer decodingkey-value cachingcontinual learningcachingattentionRoPE