skilldiff
skilldiff is an open-source CLI for A/B testing AI agent skills, prompts, and tool definitions. It uses LLM-as-a-Judge evaluation to compare agent configurations, with support for Gemini and Ollama-based workflows.
Overview
6 featuresskilldiff is a LLM eval & observability project. It focuses on comparing AI agent skills, prompts, and tool definitions consistently without manually judging outputs. It is built as an open-source project for AI developers building and evaluating agent workflows. The project is open source (MIT). It ships for the command line, and it can be self-hosted.
Behind skilldiff is shunvel, and it first shipped in 2026. The project is developed in the open on GitHub with 11 commits in the last 90 days. Among its 6 catalogued features are A/B testing, prompt evaluation, and tool definition testing.
Summary written by a language model from the project’s public pages.
- A/B testing
- Prompt evaluation
- Tool definition testing
- LLM judging
- Agent skill comparison
- CLI workflow
Built with & integrations
Trust & compliance
Indexing history
1What PulseGate has recorded for this listing
- Indexed17 Aug · 11:10 UTCskilldiff verified via PyPI Bulk EnumeratorSource: PyPI Bulk Enumerator · Open
Frequently asked questions about skilldiff
- What is skilldiff?
- Skilldiff focuses on comparing AI agent skills, prompts, and tool definitions consistently without manually judging outputs. It is catalogued under LLM eval & observability on PulseGate.
- Who is skilldiff for?
- skilldiff is an open-source project built for AI developers building and evaluating agent workflows.
- Is skilldiff free?
- Yes — skilldiff is open source under the MIT license and free to use.
- What platforms does skilldiff run on?
- skilldiff runs on the command line. It can also be self-hosted.
- Is skilldiff still maintained?
- The GitHub repository shows 11 commits in the last 90 days.
- What are alternatives to skilldiff?
- Similar projects tracked by PulseGate include skillstat, skillfed, and skilldex.skillstatskillfedskilldex
- Who develops skilldiff?
- skilldiff is developed by shunvel.
- How long has skilldiff been around?
- skilldiff first shipped in 2026.
Similar projects
Closest matches by what these projects do