llmeter is an open-source CLI tool that enables developers and researchers to profile the latency and throughput of large language models (LLMs). It provides cross-platform support for performance testing and optimization of AI workloads.
In the LLM eval & observability space, llmeter takes a focused approach. It focuses on measuring and profiling the latency and throughput of large language models for performance optimization. It is built as an open-source project for AI developers and researchers. llmeter is open source under the Apache-2.0 license. It runs on the command line.
awslabs builds and maintains llmeter, and it first shipped in 2024. Development happens publicly on GitHub with 40 stars and 105 commits in the last 90 days. Among its 5 catalogued features are latency profiling, throughput measurement, and cross-platform.
Summary written by a language model from the project’s public pages.
What PulseGate has recorded for this listing
Closest matches by what these projects do