llm-benchmark-runner
PulseGate's liveness check found it on 13 Sep 2026; it is registered on GitHub and PyPI and has been in the index since 28 Jun 2026. How this is checked
llm-benchmark-runner is an open-source CLI tool that allows developers and researchers to benchmark the inference latency and throughput of large language model APIs. It supports OpenAI-compatible endpoints and provides detailed performance metrics for model evaluation.
Inferred · not functionally tested
Overview
5 featuresPurpose: Measuring and comparing the latency and performance of large language model inference APIs for developers and researchers.
Inferred · not functionally tested
Audience: AI researchers and developers
Inferred · not functionally tested
Functions: analytics, monitoring
Inferred · not functionally tested
Interfaces: API: indicated (inferred, not tested) · MCP: unknown · CLI: indicated (inferred, not tested) · Self-hosting: unknown
Recorded constraints: pricing: open_source · license: MIT · platforms: CLI · deployment: cli
Constraint provenance is unknown; confirm requirements with the publisher.
Record sources: github.com. These links do not verify the individual claims.
In the LLM evaluation & benchmarks space, llm-benchmark-runner takes a focused approach. Inferred · not functionally tested: It focuses on measuring and comparing the latency and performance of large language model inference APIs for developers and researchers. Inferred · not functionally tested: llm-benchmark-runner is an open-source project aimed at AI researchers and developers. Basis unknown · not verified: The project is open source (MIT). Basis unknown · not verified: llm-benchmark-runner is available on the command line.
Behind llm-benchmark-runner is kuhung, and it first shipped in 2026. Development happens publicly on GitHub with 29 commits in the last 90 days. Inferred · not functionally tested: Key capabilities include LLM benchmarking, latency measurement, and API compatibility. Inferred · not functionally tested: Catalogued interfaces include a public API.
Summary written by a language model from the project’s public pages.
Tasks: Inferred · not functionally tested
- LLM benchmarking
- Latency measurement
- API compatibility
- CLI interface
- Performance metrics
Topics: Inferred · not functionally tested
Built with & integrations
Trust & compliance
Indexing history
What PulseGate has recorded for this listing
Frequently asked questions about llm-benchmark-runner
- What is llm-benchmark-runner?
- Inferred · not functionally tested: Llm-benchmark-runner focuses on measuring and comparing the latency and performance of large language model inference APIs for developers and researchers. It is catalogued under LLM evaluation & benchmarks on PulseGate.
- Who should use llm-benchmark-runner?
- Inferred · not functionally tested: llm-benchmark-runner is an open-source project built for AI researchers and developers.
- Is llm-benchmark-runner free?
- Basis unknown · not verified: Yes — llm-benchmark-runner is open source under the MIT license and free to use.
- What platforms does llm-benchmark-runner run on?
- Basis unknown · not verified: llm-benchmark-runner runs on the command line.
- Is llm-benchmark-runner still maintained?
- PulseGate's liveness check found it on 13 Sep 2026. Its GitHub repository shows 29 commits in the last 90 days.
- What projects are similar to llm-benchmark-runner?
- Similar projects tracked by PulseGate include bench-my-llm, benchrig, and llm-speed.bench-my-llmbenchrigllm-speed
- Who makes llm-benchmark-runner?
- llm-benchmark-runner is developed by kuhung.
- How long has llm-benchmark-runner been around?
- llm-benchmark-runner first shipped in 2026.
Similar projects
Closest matches by what these projects do