openagent-eval
openagent-eval is an open-source command-line framework designed for evaluating Retrieval-Augmented Generation (RAG) systems and AI agents. It provides tools and metrics for assessing LLM-based workflows, making it useful for AI researchers and developers who need to benchmark and analyze agent performance.
Open SourceApache-2.0CLI
2stars
1fork
6features
2026since
Overview
6 featuresIn the LLM eval & observability space, openagent-eval takes a focused approach. It focuses on evaluating the performance and effectiveness of RAG systems and AI agents efficiently. It is built as an open-source project for AI researchers and developers. The project is open source (Apache-2.0). It runs on the command line.
OpenAgentHQ builds and maintains openagent-eval, and it first shipped in 2026. Development happens publicly on GitHub with 67 commits in the last 90 days. Key capabilities include CLI interface, RAG evaluation, and agent benchmarking.
Summary written by a language model from the project’s public pages.
- CLI interface
- RAG evaluation
- Agent benchmarking
- Custom metrics
- Open-source
- Apache-2.0 license
Tags
rag-evaluationcli-frameworkai-metricsopen-source-toolingagent-testing
Built with & integrations
Runs on
CLI
Trust & compliance
License
Apache-2.0
Verified signals
Indexing history
1What PulseGate has recorded for this listing
Frequently asked questions about openagent-eval
- What does openagent-eval do?
- Openagent-eval focuses on evaluating the performance and effectiveness of RAG systems and AI agents efficiently. It is catalogued under LLM eval & observability on PulseGate.
- Who should use openagent-eval?
- openagent-eval is an open-source project built for AI researchers and developers.
- Does openagent-eval have a free plan?
- Yes — openagent-eval is open source under the Apache-2.0 license and free to use.
- What platforms does openagent-eval run on?
- openagent-eval runs on the command line.
- Is openagent-eval still maintained?
- The GitHub repository shows 67 commits in the last 90 days.
- What are alternatives to openagent-eval?
- Similar projects tracked by PulseGate include evalite, agentsec-eval, and agent-eval.evaliteagentsec-evalagent-eval
- Who develops openagent-eval?
- openagent-eval is developed by OpenAgentHQ.
- How long has openagent-eval been around?
- openagent-eval first shipped in 2026.
Similar projects
Closest matches by what these projects do