Recent advancements in large language model (LLM) adaptation are addressing the challenges of evolving domains and the need for efficient real-time updates. Techniques like parameter-efficient adaptation frameworks are emerging, allowing models to adjust to specific tasks without extensive retraining, thus reducing computational costs. Test-time adaptation methods, such as many-shot prompting, are being explored to enhance model behavior on-the-fly, although their effectiveness varies significantly depending on the task and selection strategy. Furthermore, approaches like Online Domain-aware Decoding are designed to tackle concept drift by enabling models to adapt continuously to new information and changing contexts. This shift towards more dynamic adaptation strategies is critical for commercial applications, as businesses increasingly require LLMs that can maintain accuracy and relevance in rapidly changing environments. Overall, the field is moving towards creating more resilient and adaptable systems that can seamlessly integrate new knowledge while preserving existing capabilities.
Using roughly 48 execution-verified HumanEval training solutions, tuning a single initial state matrix per recurrent layer, with zero inference overhead, outperforms LoRA by +10.8 pp (p < 0.001) on Hu...
Speculative decoding accelerates LLM inference but suffers from performance degradation when target models are fine-tuned for specific domains. A naive solution is to retrain draft models for every ta...
Large Language Models (LLMs) generalize across tasks via reusable representations and flexible reasoning, yet remain brittle in real deployment under evolving tasks and continual distribution shift. A...
LLM agents that store knowledge as natural language suffer steep retrieval degradation as condition count grows, often struggle to compose learned rules reliably, and typically lack explicit mechanism...
Test-time adaptation enables large language models (LLMs) to modify their behavior at inference without updating model parameters. A common approach is many-shot prompting, where large numbers of in-c...
Adapting large language models (LLMs) to new languages is an expensive and opaque process. Understanding how language models acquire new languages and multilingual abilities is key to achieve efficien...
LLMs are typically fine-tuned offline on domain-specific data, assuming a static domain. In practice, domain knowledge evolves continuously through new regulations, products, services, and interaction...
Temporal distribution shifts are pervasive in real-world deployments of Large Language Models (LLMs), where data evolves continuously over time. While Temporal Domain Generalization (TDG) seeks to mod...
Large Language Models (LLMs) remain heavily centered on English, with limited performance in low-resource languages. Existing adaptation approaches, such as continual pre-training, demand significant ...
Large language models fragment Kazakh text into many more tokens than equivalent English text, because their tokenizers were built for high-resource languages. This tokenizer tax inflates compute, sho...