llm-benchmark-runner
llm-benchmark-runner is an open-source CLI tool that allows developers and researchers to benchmark the inference latency and throughput of large language model APIs. It supports OpenAI-compatible endpoints and provides detailed performance metrics for model evaluation.
Open SourceMITCLI
1star
5features
2026since
Overview
5 featuresIn the AI space, llm-benchmark-runner takes a focused approach. It focuses on measuring and comparing the latency and performance of large language model inference APIs for developers and researchers. llm-benchmark-runner is an open-source project aimed at AI researchers and developers. The project is open source (MIT). llm-benchmark-runner is available on the command line.
Behind llm-benchmark-runner is kuhung, and it first shipped in 2026. Development happens publicly on GitHub with 29 commits in the last 90 days. Key capabilities include LLM benchmarking, latency measurement, and API compatibility.
Summary written by a language model from the project’s public pages.
- LLM benchmarking
- Latency measurement
- API compatibility
- CLI interface
- Performance metrics
Tags
llm-benchmarklatency-testingapi-performance
AI capabilities
Text
Weights: Open
Built with & integrations
Framework
next
Runs on
CLI
Trust & compliance
License
MIT
Verified signals
Indexing history
1What PulseGate has recorded for this listing
- Indexed28 Jun · 07:52 UTCllm-benchmark-runner verified against its public sourceSource: PulseGate · Open
Frequently asked questions about llm-benchmark-runner
- What is llm-benchmark-runner?
- Llm-benchmark-runner focuses on measuring and comparing the latency and performance of large language model inference APIs for developers and researchers. It is catalogued under AI on PulseGate.
- Who should use llm-benchmark-runner?
- llm-benchmark-runner is an open-source project built for AI researchers and developers.
- Is llm-benchmark-runner free?
- Yes — llm-benchmark-runner is open source under the MIT license and free to use.
- What platforms does llm-benchmark-runner run on?
- llm-benchmark-runner runs on the command line.
- Is llm-benchmark-runner still maintained?
- The GitHub repository shows 29 commits in the last 90 days.
- What projects are similar to llm-benchmark-runner?
- Similar projects tracked by PulseGate include llm-speed, inferbench-cli, and BenchLoop.llm-speedinferbench-cliBenchLoop
- Who makes llm-benchmark-runner?
- llm-benchmark-runner is developed by kuhung.
- How long has llm-benchmark-runner been around?
- llm-benchmark-runner first shipped in 2026.
Similar projects
Closest matches by what these projects do