agchk is an open-source CLI tool that audits the architecture and health of AI agent systems and LLM-integrated projects. It helps developers and auditors ensure the security and robustness of their AI deployments. Below are 27 llm eval & observability apps with similar functionality to agchk, matched by what each product actually does — not ranked or scored. Explore each to find the closest fit for your use case.
ai-agentguard is an open-source CLI tool that monitors AI coding agents for security threats such as remote code execution, MCP poisoning, and API key theft. It is designed for developers working with AI agent frameworks to enhance security.
agent-audit-kit is an open-source CLI tool that scans MCP-connected AI agent pipelines for security vulnerabilities. It helps AI security engineers ensure the safety and compliance of agent-based workflows.
aiagenthub-cli is an open-source command-line tool for running and managing AI agents. It supports agent registry, Docker integration, and automation, targeting developers building or orchestrating AI agent workflows.
agent-probe-ai is an open-source CLI tool for adversarial resilience testing of AI agents. It evaluates agent robustness against attacks like memory poisoning and tool misuse, helping AI researchers and security engineers identify vulnerabilities.
agtag is an open-source command-line tool that facilitates agent-to-agent communication for developers building distributed or agent-based systems. It provides a simple CLI interface for sending and receiving messages between agents.
agentaudit-scanner is an open-source CLI tool that scans AI agent skills, rule files, and MCP configurations for security risks like prompt injections and hidden instructions. It is designed for AI security engineers and developers to ensure safe agent deployments.
agentgrep is an open-source CLI tool that enables users to search and review local AI agent prompts and history from platforms like Codex, Claude, Gemini, and VS Code. It provides read-only access for efficient prompt management and auditing.
Fetch and deploy AI agent resources from git repos to local tool directories
agent-debug is an open-source CLI tool for diagnosing failures in AI agent workflows. It provides root cause analysis and actionable fix suggestions, helping AI developers and researchers debug and improve their agent systems.
agent-id-cli is an open-source command-line interface for managing identities of AI agents. It allows developers to create, update, and organize agent identities efficiently. Suitable for AI developers and teams working with multiple agents.
agentlint is an open-source CLI tool that provides real-time quality guardrails and linting for AI coding agents. It helps developers maintain code quality and safety by integrating with agentic coding workflows and enforcing best practices.
AI Dev Team -- multi-agent coding pipeline where Claude codes and other models advise, debug, review, and plan. Built on OpenRouter.
Offline AI agent bill of materials and attack surface scanner
CLI tool for deploying and managing AI coding agent configurations (MCP servers, skills, instructions) across multiple providers.
agnt is an open-source MCP server designed for AI coding agents. It provides process management, reverse proxy with traffic logging, browser instrumentation, and a sketch mode for agent development. Ideal for developers building or managing AI agent frameworks.
AI-native quality gate for agent-generated code — scan, audit, auto-fix, trend
Score any URL for AI-agent readiness — llms.txt, JSON-LD, AI-bot robots.txt, canonical, MCP, meta, sitemap.
agentsec-eval is an open-source CLI framework for evaluating the security of AI agents. It provides adversarial test runners, server-side audits, and scoring mechanisms to help researchers and developers identify vulnerabilities such as prompt injection and improve agent robustness.
agentguardCI is an open-source command-line tool that provides contract testing for tool-using AI agents. It integrates with CI pipelines to automatically evaluate and validate agent behaviors, helping developers ensure reliability and correctness of their AI systems.
agentbench-cli is an open-source command-line tool that allows AI developers and researchers to evaluate, test, and scan the behavior and safety of AI agents. It provides automated checks and reporting for agent performance and compliance.
agentcheck-sdk is an open-source Python SDK that helps developers record and manage the permissions and actions of AI agents. It is designed for AI developers who need to track agent delegation and authorization.
qa-agent is an open-source CLI tool designed for agentic exploratory QA testing of web applications. It integrates with Playwright, supports accessibility checks, and automates complex testing workflows, making it ideal for QA engineers and developers seeking robust testing automation.
A suite of AI-powered command-line tools for text correction, audio transcription, and voice assistance.
ai-immune is an open-source CLI tool that provides security features for AI agents, including detection of over 285 threat patterns, PII sanitization, and OWASP compliance. It is designed for AI developers and security engineers to safeguard AI systems against vulnerabilities.
An AI agent interrogation framework for identifying attack surface.
agenthint is an open-source CLI utility that detects the presence of various AI agent runtimes and adapts command-line output accordingly. It is useful for developers integrating or debugging AI agent workflows in their development environments.
Agent testing SDK. Instrument, assert, classify.
