LLMCap

LLMCap is an infrastructure tool designed to enforce hard dollar spending caps on API calls to large language model (LLM) providers. It addresses the problem of unexpected or excessive billing by stopping API calls the instant a user-defined budget is reached, rather than simply issuing alerts. When a specified cap—such as $50—is hit, LLMCap returns an HTTP 429 response before the token is consumed, ensuring that no additional charges are incurred and eliminating the risk of surprise bills.

The tool supports six major LLM providers, including Anthropic, OpenAI, Google Gemini, Mistral, Cohere, and AWS Bedrock. Integration requires only a single line change to the API client, redirecting requests through the LLMCap proxy. It supports all major SDKs and works with streaming responses, closing the connection and sending a 429 event if the cap is exceeded mid-stream. Users can define daily, monthly, or per-key dollar limits through a dashboard, with per-model granularity available. LLMCap also offers spend anomaly detection, sending configurable Slack alerts when daily spend exceeds a user-set multiplier of the recent average. Cache efficiency is tracked, with alerts for significant drops in cache hit rate, specifically for Anthropic prompts. Deploy cost attribution is enabled by tagging requests, allowing users to break down costs by deployment version in the dashboard.

LLMCap is accessible through multiple interfaces. There is a VS Code extension that displays live spend, usage, and blocked count directly in the status bar. A command-line interface is available via PyPI, supporting macOS, Linux, and Windows, and a Windows tray app provides always-visible live spend statistics and quick access to key actions. The tool is designed to be easy to set up, with a five-minute integration process and no need to store provider API keys; keys are passed through per request and immediately discarded, with only the LLMCap proxy key being stored (hashed with bcrypt).

Pricing is offered on a subscription basis with a three-day free trial. The Starter plan costs $19 per month and includes support for two API keys, all six providers, daily and monthly caps, cache hit rate monitoring, deploy cost attribution, a 30-day audit log, single-user access, and email support. The Pro plan is $49 per month, offering unlimited API keys, per-key and per-model caps, Slack spend anomaly alerts, a 90-day audit log, support for up to five users, and priority support. Self-hosting is not currently available but is mentioned as a future roadmap item.