Evaluation harness for real headless CLI agents - reproducible multi-turn scenarios, rule + LLM scoring, cross-agent comparison Below are 39 llm eval & observability apps with similar functionality to agent-belt, matched by what each product actually does — not ranked or scored. Explore each to find the closest fit for your use case.
Unified CLI for the Agent Quality Toolkit (agentmd, coderace, agentlint, agentreflect)
agentbench-cli is an open-source command-line tool that allows AI developers and researchers to evaluate, test, and scan the behavior and safety of AI agents. It provides automated checks and reporting for agent performance and compliance.
Contract-based testing framework for LLM agents — hybrid metrics (objective + LLM-as-judge + human), multi-agent and A2A protocol support, deterministic record/replay envelope.
A lightweight agent harness built with Textual and a configurable multi-provider backend
Agent evaluation toolkit
Security harness for AI coding agents (Claude Code, Codex CLI, etc.) — mitmproxy payload inspection + TOML policy control.
A suite of AI-powered command-line tools for text correction, audio transcription, and voice assistance.
Language-agnostic correctness checker for scientific numerical code
Lint, score, and optimize AI agent configuration files (CLAUDE.md, AGENTS.md, Cursor rules)
CLI-Toolkit für AI-Agent-Infrastruktur: .agent-Ordner, Tool-Discovery, Credential-Management
Agent testing SDK. Instrument, assert, classify.
Chaos proxy for testing LLM agent resilience — inject faults like latency, errors, and bad responses. Supports OpenAI, Anthropic, and MCP.
Deterministic MCP-first browser automation runtime for coding agents
A CLI-driven virtual filesystem for AI agents
Self-describing local inbox for coding agents
AgentX - Native compiled Python agent framework
Pick the right tools for an agent task. Boring baseline. Open dataset.
Multi-agent orchestrator for AI coding tools
Agent Brain CLI - Command-line interface for managing AI agent memory and knowledge retrieval
Capability-first Codex control plane for long-running developer workflows.
Lighthouse for AI agent development environments
agent-security-harness is an open-source CLI tool that automates security and compliance testing for AI agent systems. It supports MCP and other protocols, providing a comprehensive suite of tests aligned with industry standards for developers and security teams.
Local AI-agent usage analytics and persona archetypes
Autonomous Offensive Security Intelligence - AI-powered penetration testing
agentsec-eval is an open-source CLI framework for evaluating the security of AI agents. It provides adversarial test runners, server-side audits, and scoring mechanisms to help researchers and developers identify vulnerabilities such as prompt injection and improve agent robustness.
AI-native quality gate for agent-generated code — scan, audit, auto-fix, trend
Multi-turn agent benchmarking with ACP — run any agent, any model, any provider.
Translate canonical coding-agent definitions into platform-native agent artifacts.
Bridge OpenAI tools to Claude Code SDK, Codex CLI, and OpenRouter
A Langgraph agent API server that implements Langgraph agent's web capabilities and can interact with chatbot
Clone AI agent configurations into any project
Contrast Security's agent for Python web frameworks
Agent runner for clawmeets multi-agent collaboration
Generate runnable AI agent projects from markdown specs in the agent-deployments repo.
Agent discovery tool for networks
A multi-agent AI coding system with built-in RAG — local or cloud, CLI or API.
AI agent for bare-metal hardware detection and OS installation — conversationally
Thin CLI + MCP client for Agent-CoreX — execute any task with a single query
agent-id-cli is an open-source command-line interface for managing identities of AI agents. It allows developers to create, update, and organize agent identities efficiently. Suitable for AI developers and teams working with multiple agents.
