harnessgym is an open-source CLI framework for benchmarking, testing, and improving agent harnesses for AI code agents such as Codex. Below are 9 llm eval & observability apps with similar functionality to harnessgym, matched by what each product actually does — not ranked or scored. Explore each to find the closest fit for your use case.
Harness engineering framework for AI coding agents -- the invisible skeleton that shapes agent output
Harnessie is an open-source framework designed to provide verifiable orchestration for multi-agent AI systems, emphasizing user control and auditability. It addresses the challenges of supervising AI models by enforcing independent verification of each task, ensuring that no action is considered complete until an external verifier approves it. Sensitive data remains confined to user-controlled models, and every action—whether by AI or human—is recorded in a tamper-evident, hash-chained audit log. The tool is structured around three types of agents: an orchestrator that decomposes jobs into tasks, workers that execute tasks in isolated environments, and verifiers that independently check results without access to the worker’s process. Each step in a workflow is separated by checkpoints, and the system defaults to failing closed, meaning no change is made without explicit approval. Consent is required before any side effect, such as changing a file or running a command, and declined actions are logged rather than overridden. In cases requiring judgment, the process halts for human intervention, ensuring the final decision remains with the operator. Harnessie is brain-agnostic, supporting both local and remote models. Users can switch between model providers—including local open-source models or those accessible via OpenAI-compatible endpoints—by editing a single configuration file, without altering the underlying workflow structure. 11+ and PyYAML for installation. It can be installed via pip, pipx, uv, or brew, and does not rely on vendor-specific SDKs. The included test suite and evaluation scorecard allow users to validate the harness in a deterministic, network-free environment before integrating any API keys. 0, making its safety and operation fully transparent and auditable to users. Harnessie is suitable for developers and operators who require strict control, verifiability, and auditability in multi-agent AI workflows, particularly where sensitive data and independent verification are priorities.
OpenHarness is an open-source SDK for developers to build AI agents in code, offering composable primitives, middleware, and support for subagent hierarchies. Built on the Vercel AI SDK, it enables flexible, stateless agent development for advanced automation and agentic applications.
harness-scorecard is an open-source CLI tool that provides linting, A-F maturity grading, and security analysis for coding-agent harnesses such as Claude Code and Codex. It helps developers maintain high code quality and security standards.
superharness is an open-source CLI framework for managing session handoff between multiple AI coding agents, such as Claude Code and Codex. It is designed for developers and researchers working with autonomous agent systems.
granary-harness is an open-source harness for agent security, governance, and privilege containment. It provides tools for enforcing zero-trust principles and managing agent permissions, making it suitable for developers building secure, autonomous agent systems.
Observability tooling for agent harness sessions, imports, and reports.
oneharness-cli is an open-source command-line tool that enables developers to run multiple agentic coding harnesses non-interactively and receive standardized JSON outputs. It simplifies the orchestration of agent-based coding workflows and is distributed under the MIT license.
spice-harness is an open-source CLI tool that provides an agent harness for coding repositories, enabling developers to automate, integrate, and control agent workflows in their codebases. It is designed for developers seeking to streamline agent-based automation and integration tasks.
