Colette

Colette is an open-source, self-hosted software platform designed for retrieval-augmented generation (RAG) and large language model (LLM) serving, with a focus on technical document analysis. It is intended for users and organizations that need to search and interact locally with a wide variety of technical documents—including formats such as doc, pptx, pdf, jpg, and png—without exposing sensitive content to external APIs or online LLM services.

A key distinguishing feature of Colette is its Vision-RAG (V-RAG) capability, which processes and analyzes all documents as images. This approach enables the preservation and handling of visual elements within documents, such as images, figures, schemas, visual highlights, and complex layouts. By leveraging vision and multimodal language models, Colette aims to provide deeper analysis of documents that are primarily designed for human readers. In addition to its vision-based retrieval, Colette also supports text-based RAG, offering unstructured text extraction, text embeddings, and integration with common LLMs.

The platform offers multi-model support, allowing users to switch between different embedding models and perform inference using various vision-language large models (VLLMs). Colette also integrates image generation capabilities through diffusers, broadening its functionality for technical documentation that includes visual content. The software is delivered as a dockerized, self-hosted solution, facilitating straightforward setup and deployment in environments where data privacy and local processing are critical.

Colette is open-source, and its documentation highlights collaboration with Jolibrain, a company specializing in AI application development for industry, which offers customized versions tailored to specific requirements. The tool is positioned as suitable for technical domains where the integrity and confidentiality of documentation must be maintained.