llm-d
llm-d is an open-source, high-performance distributed inference serving stack designed for deploying large language models on various hardware accelerators. It provides orchestration, optimization, and benchmarking tools for production ML inference, targeting ML engineers and infrastructure teams.
Open SourceApache-2.0WebCLIAPISelf-hosted
3.5kstars
553forks
5features
2025since
Overview
5 featuresIn the AI space, llm-d takes a focused approach. It simplifies and optimizes the deployment of large language models for high-performance inference on diverse hardware. llm-d is an open-source project aimed at machine learning engineers. The project is open source (Apache-2.0). It ships for the web, the command line, and API, and it can be self-hosted.
Behind llm-d is CNCF (Red Hat, Google Cloud, IBM Research, CoreWeave, NVIDIA), and it first shipped in 2025. Development happens publicly on GitHub with 3.5k stars and 530 commits in the last 90 days. Among its 5 catalogued features are distributed inference, kubernetes support, and model orchestration.
Summary written by a language model from the project’s public pages.
- Distributed inference
- Kubernetes support
- Model orchestration
- Accelerator optimization
- Benchmarking
Tags
distributed-inferencellm-servingkubernetes-ml
AI capabilities
Text
Inference: HybridWeights: Open
Built with & integrations
Runs on
BrowserCLIAPI-onlySelf-hosted
Trust & compliance
License
Apache-2.0
Verified signals
Indexing history
1What PulseGate has recorded for this listing
Frequently asked questions about llm-d
- What is llm-d?
- Llm-d simplifies and optimizes the deployment of large language models for high-performance inference on diverse hardware. It is catalogued under AI on PulseGate.
- Who is llm-d for?
- llm-d is an open-source project built for machine learning engineers.
- Is llm-d free?
- Yes — llm-d is open source under the Apache-2.0 license and free to use.
- What platforms does llm-d run on?
- llm-d runs on the web, the command line, and API. It can also be self-hosted.
- Is llm-d still active?
- The GitHub repository shows 530 commits in the last 90 days.
- What are alternatives to llm-d?
- Similar projects tracked by PulseGate include Mesh LLM, Moncef Abboud, and llm.Mesh LLMMoncef Abboudllm
- Who develops llm-d?
- llm-d is developed by CNCF (Red Hat, Google Cloud, IBM Research, CoreWeave, NVIDIA).
- When did llm-d launch?
- llm-d first shipped in 2025.
Similar projects
Closest matches by what these projects do