proofagent-sdk

ProofAgent is an open-source platform designed for rigorous evaluation of AI agents prior to deployment. It addresses the challenge of ensuring AI agents perform reliably and safely in production by employing adversarial multi-juror scoring, production log audits, artifact reviews, and generating signed readiness reports. The platform is built around the ProofAgent Harness, which implements a research-backed methodology for stress-testing AI agents in scenarios such as customer support, healthcare triage, code generation, privacy compliance, and financial advisory workflows.

Distinctively, ProofAgent conducts multi-turn adversarial conversations, scoring the entire interaction trajectory with three juror personas who reach consensus through debate or Delphi methods. This approach aims to uncover domain-specific failure modes, such as those related to compliance standards like HIPAA, PCI, SOX, and GDPR, as well as vulnerabilities to social engineering and prompt injection. The evaluation process leverages a library of 183 adversarial attack traps across 11 categories, and uses five canonical metrics: task success, hallucination resistance, safety, instruction following, and manipulation resistance. Composite attack chains are used to simulate sustained adversarial pressure over 5–7 conversational turns.

0 license and is free to use and run locally. For teams requiring additional governance and operational features, the enterprise Platform adds hosted dashboards, SOC 2 and HIPAA-ready operations, regression tracking, expert human review, and signed readiness reports. The infrastructure is compatible with various large language models through LiteLLM, including Anthropic, OpenAI, Gemini, Bedrock, Ollama, and vLLM, and supports integration with pytest for CI/CD workflows. On-premises deployment is also available.

ProofAgent is intended for engineering teams responsible for shipping AI agents into production environments where safety, compliance, and accountability are critical. The platform is documented in a published whitepaper, with all results reproducible from the open-source codebase.