Modelplane

Modelplane is an open source control plane designed for managing AI inference across diverse GPU infrastructure. It enables users to install the platform within their own environments, allowing unified operation of GPU clusters that span cloud, neocloud, and on-premise resources as a single inference fleet. The system is intended to orchestrate the deployment and ongoing operation of AI models, addressing the complexity of running inference workloads across multiple types of infrastructure.

Key features include provisioning of clusters, model placement, autoscaling of replicas, caching of model weights, and routing inference requests through a single OpenAI-compatible endpoint. Modelplane supports running any model on any serving engine and infrastructure, providing flexibility for teams to use their preferred models, serving engines, and hardware accelerators. The platform also handles policy, governance, and compliance aspects related to inference operations.

Modelplane integrates with existing inference ecosystems rather than replacing them, orchestrating models, serving engines, and infrastructure across different providers and accelerator types. It supports the composition and integration of new technologies as they emerge, aiming to give users control over how their AI inference workloads are managed and scaled.

As an open source project, Modelplane can be self-hosted and operated under the user's control. Its design is suited for organizations or teams that require a unified approach to managing AI inference across heterogeneous GPU environments, whether in the cloud, neocloud, or on-premises.