ase-python is an open-source command-line tool that simulates and tests AI agent tool calls, providing a pytest-like workflow for agent evaluation. Below are 6 llm eval & observability apps with similar functionality to ase-python, matched by what each product actually does — not ranked or scored. Explore each to find the closest fit for your use case.
selenium-python-ai-agent is an open-source framework that uses AI-powered multi-agent systems to plan, generate, and heal Selenium Python tests. It integrates with pytest and BDD workflows, targeting QA engineers and automation developers.
agentassay is an open-source Python package designed for formal regression testing of non-deterministic AI agent workflows. It provides tools for verifying the reliability and correctness of agent-based systems, supporting stochastic and CI/CD testing scenarios. Ideal for AI developers and QA engineers working with complex agent architectures.
agent-probe-ai is an open-source CLI tool for adversarial resilience testing of AI agents. It evaluates agent robustness against attacks like memory poisoning and tool misuse, helping AI researchers and security engineers identify vulnerabilities.
pat-agent is an open-source AI coding agent that operates via the command line. It automates coding tasks and workflows using LLMs, making it a valuable tool for developers who want to leverage AI for code generation and automation.
arksim is an open-source CLI tool that allows AI developers and researchers to simulate, test, and evaluate the performance of AI agents and chatbots before deploying them in production. It supports LLM evaluation and provides insights into agent behavior and reliability.
Agent testing SDK. Instrument, assert, classify.
