trajectory-judge
trajectory-judge is an open-source Python library that evaluates LLM agent trajectories. It measures cases where an agent arrives at the correct answer but through flawed or unintended reasoning that standard LLM judges often miss. The package helps researchers and developers better calibrate and debug autonomous agents by providing more nuanced evaluation metrics beyond final-answer correctness.
Open SourceMITCLIAPI
Overview
4 featurestrajectory-judge is a LLM eval & observability project. It focuses on evaluating whether LLM agents followed the correct reasoning path even when they reach the right final answer. trajectory-judge is an open-source project aimed at AI researchers and developers. The project is open source (MIT). It ships for the command line and API.
Behind trajectory-judge is Hadi Mohammadi, and it first shipped in 2026. The project is developed in the open on GitHub with 38 commits in the last 90 days. Among its 4 catalogued features are Trajectory Evaluation, LLM-as-Judge, and Calibration Metrics.
Summary written by a language model from the project’s public pages.
- Trajectory Evaluation
- LLM-as-Judge
- Calibration Metrics
- Agent Debugging
Tags
llm-evaluationagent-trajectoryllm-as-judgecalibrationagent-debugging
AI capabilities
Text
Inference: Cloud API
Built with & integrations
Runs on
CLIAPI-only
Trust & compliance
License
MIT
Verified signals
Indexing history
1What PulseGate has recorded for this listing
Frequently asked questions about trajectory-judge
- What is trajectory-judge?
- Trajectory-judge focuses on evaluating whether LLM agents followed the correct reasoning path even when they reach the right final answer. It is catalogued under LLM eval & observability on PulseGate.
- Who is trajectory-judge for?
- trajectory-judge is an open-source project built for AI researchers and developers.
- Does trajectory-judge have a free plan?
- Yes — trajectory-judge is open source under the MIT license and free to use.
- What platforms does trajectory-judge run on?
- trajectory-judge runs on the command line and API.
- Is trajectory-judge still active?
- The GitHub repository shows 38 commits in the last 90 days.
- Who develops trajectory-judge?
- trajectory-judge is developed by Hadi Mohammadi.
- How long has trajectory-judge been around?
- trajectory-judge first shipped in 2026.
- Is trajectory-judge open source?
- Yes — trajectory-judge is open source under the MIT license, developed on GitHub.
Similar apps
Closest matches by what these projects do