docpipe

docpipe is an open-source platform for processing and querying documents with retrieval-augmented generation (RAG) methods, designed to work with user-managed databases and large language models. It allows users to parse, extract, ingest, and query information from documents such as PDFs, DOCX files, and images, supporting workflows that convert files to markdown, extract structured entities using LLMs, and store vector embeddings in a Postgres database. The tool supports multiple RAG strategies and offers grounded answers with source citations when querying ingested documents.

The platform is built around composable pipelines, enabling independent or end-to-end use of its four main stages: parsing, extraction, ingestion, and RAG querying. Users can select from runtime presets—such as fast, balanced, quality, or agents—or configure explicit parsers, chunkers, and rerankers. docpipe integrates with plugins and supports profiles for different workflows, and it is compatible with pgvector for vector storage. Observability features are included via health checks, Prometheus metrics, and OTEL integration.

docpipe can be used as a local SDK, a shared internal API server, or deployed via Docker images, allowing flexible integration into various development environments. It provides both a command-line interface and an API, with endpoints for managing profiles, plugins, ingestion, querying, agent discovery, and monitoring. Each application connects to its own Postgres instance, and embedding providers and models can be specified per workflow. 10+, is distributed under the MIT license, and is available on PyPI and GitHub.

This tool is suited for developers and teams seeking to build or deploy document parsing and RAG pipelines within their own infrastructure, leveraging customizable workflows and direct database integration.