pyinferencemanager
pyinferencemanager is a Python library that acts as an intelligent orchestrator for AI inference workloads. It provides multi-provider routing across services like OpenAI, Anthropic, and local models (Ollama), along with semantic caching, dynamic routing, cost optimization, budget enforcement, and hardware-aware scheduling. Designed for developers building LLM-powered applications, it simplifies managing distributed inference with built-in tracing and optimization features.
Open SourceMITCLIAPI
Overview
7 featurespyinferencemanager is an AI & ML project. It focuses on managing and routing AI inference workloads across multiple LLM providers while controlling costs, enforcing budgets, and optimizing performance. pyinferencemanager is an open-source project aimed at developers. The project is open source (MIT). It runs on the command line and API.
Behind pyinferencemanager is Mullassery, and it first shipped in 2026. The project is developed in the open on GitHub with 15 commits in the last 90 days. Among its 7 catalogued features are multi-provider routing, semantic caching, and cost optimization. It exposes integrations via a public API.
Summary written by a language model from the project’s public pages.
- Multi-provider routing
- Semantic caching
- Cost optimization
- Budget enforcement
- Dynamic routing
- Distributed tracing
- Hardware aware scheduling
Tags
llm-orchestratormulti-llm-routingsemantic-cachingcost-optimizationinference-manager
AI capabilities
TextMultimodal
Inference: Hybrid
Built with & integrations
AI providers
anthropicopenailocal_oss
Connectors
API
Runs on
CLIAPI-only
Trust & compliance
License
MIT
Verified signals
Indexing history
1What PulseGate has recorded for this listing
- Indexed22 Jul · 02:45 UTCpyinferencemanager verified against its public sourceSource: PulseGate · Open
Frequently asked questions about pyinferencemanager
- What does pyinferencemanager do?
- Pyinferencemanager focuses on managing and routing AI inference workloads across multiple LLM providers while controlling costs, enforcing budgets, and optimizing performance. It is catalogued under AI & ML on PulseGate.
- Who is pyinferencemanager for?
- pyinferencemanager is an open-source project built for developers.
- Does pyinferencemanager have a free plan?
- Yes — pyinferencemanager is open source under the MIT license and free to use.
- What platforms does pyinferencemanager run on?
- pyinferencemanager runs on the command line and API.
- Is pyinferencemanager still maintained?
- The GitHub repository shows 15 commits in the last 90 days.
- What are alternatives to pyinferencemanager?
- Similar projects tracked by PulseGate include Cliprise, chi-ai, and AI DocuChat.Cliprisechi-aiAI DocuChat
- Who develops pyinferencemanager?
- pyinferencemanager is developed by Mullassery.
- How long has pyinferencemanager been around?
- pyinferencemanager first shipped in 2026.
Also in AI & ML
Same category — not a similarity match