LeMans
LeMans is an open-source Ruby harness for running agent benchmarks. It helps developers execute repeatable evaluations and compare agent performance during development.
Overview
5 featuresIn the LLM evaluation & benchmarks space, LeMans takes a focused approach. It focuses on running repeatable benchmarks to measure and compare AI agent performance. It is built as an open-source project for AI developers and Ruby developers building agent systems. LeMans is open source under the Open Source license. LeMans is available on the command line, and it can be self-hosted.
Rails builds and maintains LeMans. Among its 5 catalogued features are agent benchmarks, benchmark harness, and ruby tooling.
Summary written by a language model from the project’s public pages.
- Agent benchmarks
- Benchmark harness
- Ruby tooling
- Repeatable evaluations
- Performance comparison
Built with & integrations
- openai
- bgpt- in the HTML
- multiple
- bopenrouter in the HTML
Trust & compliance
Indexing history
2What PulseGate has recorded for this listing
Frequently asked questions about LeMans
- What is LeMans?
- LeMans focuses on running repeatable benchmarks to measure and compare AI agent performance. It is catalogued under LLM evaluation & benchmarks on PulseGate.
- Who should use LeMans?
- LeMans is an open-source project built for AI developers and Ruby developers building agent systems.
- Is LeMans free?
- Yes — LeMans is open source under the Open Source license and free to use.
- What platforms does LeMans run on?
- LeMans runs on the command line. It can also be self-hosted.
- Is LeMans still active?
- Unverified. LeMans has not been re-checked since it entered the index, so there is no finding either way — and only a positive finding would say otherwise.
- What projects are similar to LeMans?
- Similar projects tracked by PulseGate include nengok, Promptfoo, and inferbench-cli.nengokPromptfooinferbench-cli
- Who makes LeMans?
- LeMans is developed by Rails.
- Is LeMans open source?
- Yes — LeMans is open source under the Open Source license.
Also in LLM evaluation & benchmarks
Same category — not a similarity match