agenttelemetry is an open-source Python package that provides OpenTelemetry-based observability for autonomous AI agent systems. Below are 7 llm eval & observability apps with similar functionality to agenttelemetry, matched by what each product actually does — not ranked or scored. Explore each to find the closest fit for your use case.
agentmesh-observability is an open-source observability toolkit for Agent OS, providing integration with OpenTelemetry, Prometheus, and Grafana. It helps developers monitor, trace, and analyze production deployments of agent-based systems efficiently.
agentao is an open-source governed agent runtime designed for running AI agents locally with a focus on privacy and embeddability. It supports multimodal agents, integrates with MCP, and is suitable for developers building private, local-first AI solutions.
agent-diagnostics is an open-source framework for analyzing and annotating the behavior and reliability of coding agents. It provides tools for benchmarking, taxonomy creation, and structured evaluation, helping researchers and developers understand agent performance.
agentu is an open-source AI agent runtime designed for developers to run autonomous agents with tool isolation, self-correction, and permission scoping. It provides a CLI interface and is suitable for building secure, extensible AI workflows.
agent-probe-ai is an open-source CLI tool for adversarial resilience testing of AI agents. It evaluates agent robustness against attacks like memory poisoning and tool misuse, helping AI researchers and security engineers identify vulnerabilities.
agentops-cockpit is an open-source, production-grade platform for managing AI agent operations via the command line. It is designed for AI engineers and developers who need robust tools for deploying and monitoring AI agents.
traceagently is an open-source CLI tool that provides real-time observability for AI agents, allowing developers to trace every thought, tool call, and error as they occur. It supports integration with popular LLM frameworks and is aimed at AI developers.
