journeyman-bench
journeyman-bench is an open-source benchmark for evaluating the process quality of LLM agents. It assesses agent workflows and tool use, and uses calibrated LLM-as-a-judge evaluations for researchers and developers.
Overview
6 featuresIn the Agent evaluation & testing space, journeyman-bench takes a focused approach. It focuses on evaluating how LLM agents work, including their tool use and decision-making process, rather than only checking final outcomes. It is built as an open-source project for AI researchers and developers evaluating LLM agents. The project is open source (Apache-2.0). It ships for the command line, and it can be self-hosted.
It is developed by codechu, and it first shipped in 2026. The project is developed in the open on GitHub with 87 commits in the last 90 days. Among its 6 catalogued features are agent benchmarking, process evaluation, and tool-use grading.
Summary written by a language model from the project’s public pages.
- Agent benchmarking
- Process evaluation
- Tool-use grading
- LLM judge calibration
- Agent evaluation
- Benchmark scoring
Built with & integrations
Trust & compliance
Indexing history
1What PulseGate has recorded for this listing
- Indexed26 Aug · 08:08 UTCjourneyman-bench verified via PyPI Fresh FeedSource: PyPI Fresh Feed · Open
Frequently asked questions about journeyman-bench
- What is journeyman-bench?
- Journeyman-bench focuses on evaluating how LLM agents work, including their tool use and decision-making process, rather than only checking final outcomes. It is catalogued under Agent evaluation & testing on PulseGate.
- Who should use journeyman-bench?
- journeyman-bench is an open-source project built for AI researchers and developers evaluating LLM agents.
- Does journeyman-bench have a free plan?
- Yes — journeyman-bench is open source under the Apache-2.0 license and free to use.
- What platforms does journeyman-bench run on?
- journeyman-bench runs on the command line. It can also be self-hosted.
- Is journeyman-bench still active?
- The GitHub repository shows 87 commits in the last 90 days.
- Who develops journeyman-bench?
- journeyman-bench is developed by codechu.
- When did journeyman-bench launch?
- journeyman-bench first shipped in 2026.
- Is journeyman-bench open source?
- Yes — journeyman-bench is open source under the Apache-2.0 license, developed on GitHub.
Similar projects
Closest matches by what these projects do