Routerly.ai

Routerly.ai is a self-hosted LLM gateway for routing AI requests between an application and multiple model providers. It is described as a gateway that uses AI to decide which AI to use, with the stated aim of giving users total control while avoiding vendor lock-in and code changes when switching providers.

Its routing layer evaluates candidate models in parallel through nine configurable policies, which can favor the cheapest, fastest, healthiest, or most capable option, or combine those criteria. Routerly also supports drop-in compatibility with OpenAI and Anthropic wire formats, including full streaming, so a client can point at localhost:3000 instead of a provider URL. Beyond routing, it tracks per-token pricing, includes presets for GPT-4o, Claude, and Gemini, accepts custom pricing for any model, and enforces hourly, daily, weekly, or monthly budget caps either per project or globally. Over-budget models are filtered before routing starts. Separate bearer tokens provide project isolation with distinct model lists, routing configuration, and budget limits.

The product is aimed at developers and organizations that need reliability, cost visibility, and control over AI infrastructure. The page also names SaaS and multi-tenant deployments, local-first development, automatic cost reduction, resilience with failover, AI tools and chat UIs, and model evaluation or A/B testing as use cases. It says Routerly can sit between an app or AI tool and providers such as OpenAI, Anthropic, Gemini, Ollama, Mistral, Cohere, and xAI, as well as custom OpenAI-compatible endpoints.

Routerly is open source, built with TypeScript, runs on Docker, and is licensed under AGPL-3.0. It is offered as free and self-hosted, with no markup on API calls. A web dashboard is included for visual analytics, model management, project configuration, and user and role administration, and the product can be installed with a one-liner script on macOS, Windows, and Linux.