Filtered to language models · clear filter
This paper proposes a new method for managing the memory of large language models, called Random Attention, which improves their performance on tasks that require extended reasoning by reducing memory usage. Practitioners caring about efficient model deployment might find this approach useful.
This paper creates a platform to help large language models generate code for data pipelines, which can then be edited and used to automate data processing workflows. Practitioners might care about this because it can help reduce the time and cost of developing and maintaining these pipelines.
This paper introduces LatentPress, a method to compress conversational histories and documents into a continuous memory token format that allows language models to directly read and process the context without text reconstruction. Practitioners might care about this because it could lead to faster and more efficient language model inference.
This paper introduces Co-RL, a framework for unsupervised multi-agent reinforcement learning that enables diverse and accurate reasoning in language and vision-language models. Practitioners can use Co-RL to improve their models' ability to reason and respond without relying on expensive ground-truth supervision.
This paper proposes a new method for fine-tuning large language models (LLMs) in reinforcement learning (RL) tasks with long horizons, using evolution strategies (ES) instead of traditional backpropagation-based training. Practitioners might care because it allows for more efficient and flexible fine-tuning of LLMs, enabling them to tackle complex tasks with larger models and longer interactions.
This paper introduces AURORA-LM, a continuous-latent diffusion language model that generates text by learning a high-capacity, decodable text latent and its distribution directly. Practitioners might care because it achieves state-of-the-art performance on text generation and summarization tasks.
This paper develops a method to scale up language models by increasing their memory capacity, allowing for better performance and more efficient use of parameters. Practitioners may care about this research as it could lead to more powerful and efficient language models for applications like language translation and text generation.
This paper introduces a new attention mechanism, Declarative Attention (DA), that allows language models to more efficiently control their attention by specifying where they need to focus during generation, rather than scanning the entire context. Practitioners might care because this could lead to significant reductions in computation and memory usage, making large language models more practical for real-world applications.
This paper introduces SPADE, a self-play framework that enables language agents to learn from adaptive, self-generated environments, allowing them to improve continuously without fixed goal distributions. Practitioners might care because SPADE can lead to more robust and open-ended AI models.
This paper proposes a new method for estimating confidence in language models, called XConf, which uses the model's past experiences to inform its confidence, rather than just relying on the current inference process. Practitioners might care about this because it could lead to more reliable and trustworthy deployment of language models.
This paper introduces Agent Memory Distillation, a technique that allows small language models to learn from a larger teacher model by transferring structured knowledge through hierarchical memory. Practitioners might care about this approach because it could improve the performance of small language models in tasks that require complex decision-making.