promptdiff-eval
PulseGate's liveness check found it on 7 Oct 2026; it is registered on GitHub and PyPI and has been in the index since 7 Sep 2026. How this is checked
promptdiff-eval is an open-source Python package for asynchronous LLM prompt regression testing and evaluation. It includes RAG evaluators, security guardrails, DSPy auto-optimization, a CLI, and a Streamlit dashboard for AI engineering teams.
Inferred · not functionally tested
Overview
6 featuresPurpose: Testing and comparing LLM prompts and RAG systems reliably across regressions, quality checks, and security risks.
Inferred · not functionally tested
Audience: AI engineers and ML platform developers
Inferred · not functionally tested
Functions: analytics, monitoring
Inferred · not functionally tested
Interfaces: API: unknown · MCP: unknown · CLI: indicated (inferred, not tested) · Self-hosting: indicated (inferred, not tested)
Recorded constraints: pricing: open_source · license: MIT · platforms: CLI · deployment: cli, self_hosted
Constraint provenance is unknown; confirm requirements with the publisher.
Record sources: pypi.org · github.com. These links do not verify the individual claims.
In the Agent evaluation & testing space, promptdiff-eval takes a focused approach. Inferred · not functionally tested: It focuses on testing and comparing LLM prompts and RAG systems reliably across regressions, quality checks, and security risks. Inferred · not functionally tested: promptdiff-eval is an open-source project aimed at AI engineers and ML platform developers. Basis unknown · not verified: promptdiff-eval is open source under the MIT license. Basis unknown · not verified: It ships for the command line, and it can be self-hosted.
It is developed by latryee, and it first shipped in 2026. The project is developed in the open on GitHub with 147 commits in the last 90 days. Inferred · not functionally tested: Key capabilities include prompt regression testing, async evaluation, and RAG evaluators.
Summary written by a language model from the project’s public pages.
Tasks: Inferred · not functionally tested
- Prompt regression testing
- Async evaluation
- RAG evaluators
- Security guardrails
- DSPy optimization
- LLM judging
Topics: Inferred · not functionally tested
Built with & integrations
Trust & compliance
Indexing history
1What PulseGate has recorded for this listing
- Indexed7 Sep · 02:22 UTCpromptdiff-eval seen via PyPI Bulk EnumeratorSource: PyPI Bulk Enumerator · Open
Frequently asked questions about promptdiff-eval
- What is promptdiff-eval?
- Inferred · not functionally tested: Promptdiff-eval focuses on testing and comparing LLM prompts and RAG systems reliably across regressions, quality checks, and security risks. It is catalogued under Agent evaluation & testing on PulseGate.
- Who should use promptdiff-eval?
- Inferred · not functionally tested: promptdiff-eval is an open-source project built for AI engineers and ML platform developers.
- Is promptdiff-eval free?
- Basis unknown · not verified: Yes — promptdiff-eval is open source under the MIT license and free to use.
- What platforms does promptdiff-eval run on?
- Basis unknown · not verified: promptdiff-eval runs on the command line. It can also be self-hosted.
- Is promptdiff-eval still maintained?
- PulseGate's liveness check found it on 7 Oct 2026. Its GitHub repository shows 147 commits in the last 90 days.
- What are alternatives to promptdiff-eval?
- Similar projects tracked by PulseGate include promptdiff-cli, promptdrift-ci, and PromptEval.promptdiff-clipromptdrift-ciPromptEval
- Who develops promptdiff-eval?
- promptdiff-eval is developed by latryee.
- How long has promptdiff-eval been around?
- promptdiff-eval first shipped in 2026.
Similar projects
Closest matches by what these projects do