Doc-to-LoRA

Doc-to-LoRA is a tool designed to address the challenges of long-term memory and continual adaptation in large language models (LLMs). It enables LLMs to internalize new factual content by converting documents into LoRA adapters, which modify the model's behavior without requiring full retraining or repeated inclusion of the source document in the prompt. This approach aims to reduce the latency and memory overhead typically associated with long-context prompting or context window limitations, allowing users to ask multiple questions about an internalized document without reprocessing it each time.

The core functionality of Doc-to-LoRA involves using auxiliary modulator networks, also referred to as hypernetworks, to generate LoRA adapters directly from text inputs. When a user provides a document—such as a policy, report, or PDF—the tool creates an adapter that stores the relevant information. This enables the base model to answer queries based on the internalized knowledge, effectively simulating a persistent memory toggle. The tool is positioned as a faster and more cost-effective alternative to traditional LLM knowledge update methods, such as context distillation or full fine-tuning, which can be slow, computationally expensive, and require extensive data curation and engineering.

Doc-to-LoRA is relevant for practitioners and researchers working with LLMs who need to quickly update models with new knowledge or skills, especially in scenarios where information must be reused across multiple queries or sessions. The tool is presented alongside Text-to-LoRA, which generates LoRA adapters from task descriptions for task-specific fine-tuning, but Doc-to-LoRA specifically focuses on knowledge updates from documents.

A demonstration is referenced, showing a chat interface where users provide a context document and then interact with the model through queries that leverage the internalized adapter.