skill-eval-harness

Purpose: Standardizing and automating the evaluation of agent skills across benchmarks and variants.

Inferred · not functionally tested

Audience: AI researchers and developers evaluating agent performance

Inferred · not functionally tested

Functions: analytics

Inferred · not functionally tested

Interfaces: API: indicated (inferred, not tested) · MCP: unknown · CLI: indicated (inferred, not tested) · Self-hosting: unknown

Recorded constraints: pricing: open_source · license: MIT · platforms: CLI · deployment: cli

Constraint provenance is unknown; confirm requirements with the publisher.

Record sources: pypi.org · github.com. These links do not verify the individual claims.

In the Agent evaluation & testing space, skill-eval-harness takes a focused approach. Inferred · not functionally tested: It focuses on standardizing and automating the evaluation of agent skills across benchmarks and variants. Inferred · not functionally tested: It is built as an open-source project for AI researchers and developers evaluating agent performance. Basis unknown · not verified: skill-eval-harness is open source under the MIT license. Basis unknown · not verified: It runs on the command line.

Behind skill-eval-harness is adewale, and it first shipped in 2026. The project is developed in the open on GitHub with 54 stars and 76 commits in the last 90 days. Inferred · not functionally tested: Key capabilities include skill evaluation, paired variants, and holdout splits. Inferred · not functionally tested: Catalogued interfaces include a public API.

Summary written by a language model from the project’s public pages.