Skip to content
Back to the index

llamathon

PyPIInfrastructure

PulseGate's liveness check found it on 14 Sep 2026; it is registered on GitHub and PyPI and has been in the index since 24 Jul 2026. How this is checked

Llamathon is a Python benchmarking tool for LLM inference servers. It orchestrates tests using llama-benchy with automatic hardware and model detection, realistic workload generation, and produces charts and detailed reports. It supports popular backends including Ollama, vLLM, and llama.cpp.

Inferred · not functionally tested

Open SourceMITCLISelf-hosted
Visit PyPI
1star
5features
2026since

Overview

5 features

Purpose: Accurately measuring and reporting the real-world LLM inference performance and capacity of a server.

Inferred · not functionally tested

Audience: ML engineers and infrastructure teams

Inferred · not functionally tested

Functions: analytics, monitoring

Inferred · not functionally tested

Interfaces: API: unknown · MCP: unknown · CLI: indicated (inferred, not tested) · Self-hosting: indicated (inferred, not tested)

Recorded constraints: pricing: open_source · license: MIT · platforms: CLI · deployment: cli, self_hosted

Constraint provenance is unknown; confirm requirements with the publisher.

Record sources: pypi.org · github.com. These links do not verify the individual claims.

In the Inference & model serving space, llamathon takes a focused approach. Inferred · not functionally tested: Accurately measuring and reporting the real-world LLM inference performance and capacity of a server. Inferred · not functionally tested: It is built as an open-source project for ML engineers and infrastructure teams. Basis unknown · not verified: llamathon is open source under the MIT license. Basis unknown · not verified: It runs on the command line, and it can be self-hosted.

llamathon first shipped in 2026. The project is developed in the open on GitHub with 15 commits in the last 90 days. Inferred · not functionally tested: Among its 5 catalogued features are LLM Benchmarking, Inference Testing, and auto-detection.

Summary written by a language model from the project’s public pages.

Tasks: Inferred · not functionally tested

  • LLM Benchmarking
  • Inference Testing
  • Auto-detection
  • Performance Reports
  • Charts

Topics: Inferred · not functionally tested

Tags
llm-benchmarkinference-benchollamavllmllama-cpp
AI capabilities
Text
Inference: Local

JSON profile · Text profile · Access guide

Built with & integrations

Runs on
CLISelf-hosted

Trust & compliance

License
MIT
Public signals
HTTPSOpen SourceGitHub · ★ 1Active maintenance

Indexing history

1

What PulseGate has recorded for this listing

  1. Indexed24 Jul · 08:21 UTC
    llamathon seen via PyPI Fresh Feed
    Source: PyPI Fresh Feed · Open

Frequently asked questions about llamathon

What is llamathon?
Inferred · not functionally tested: Accurately measuring and reporting the real-world LLM inference performance and capacity of a server. It is catalogued under Inference & model serving on PulseGate.
Who is llamathon for?
Inferred · not functionally tested: llamathon is an open-source project built for ML engineers and infrastructure teams.
Does llamathon have a free plan?
Basis unknown · not verified: Yes — llamathon is open source under the MIT license and free to use.
What platforms does llamathon run on?
Basis unknown · not verified: llamathon runs on the command line. It can also be self-hosted.
Is llamathon still active?
PulseGate's liveness check found it on 14 Sep 2026. Its GitHub repository shows 15 commits in the last 90 days.
What projects are similar to llamathon?
Similar projects tracked by PulseGate include Localmaxxing, llm-speed, and llm-benchmark-runner.Localmaxxingllm-speedllm-benchmark-runner
How long has llamathon been around?
llamathon first shipped in 2026.
Is llamathon open source?
Basis unknown · not verified: Yes — llamathon is open source under the MIT license, developed on GitHub.

Similar projects

Closest matches by what these projects do