judge-drift-sentinel
judge-drift-sentinel is a Python package that monitors LLM evaluation pipelines. It uses a frozen set of human-labeled anchor examples to distinguish between changes in the underlying system and drift in the LLM judge's scoring behavior. This helps AI engineering teams maintain reliable evaluation signals over time.
Open SourceMITCLIAPI
Overview
4 featuresjudge-drift-sentinel is a LLM eval & observability project. It focuses on detecting whether shifts in LLM evaluation scores are caused by changes to the system under test or by drift in the LLM judge itself. It is built as an open-source project for AI engineers and LLM application developers. judge-drift-sentinel is open source under the MIT license. It runs on the command line and API.
It is developed by Homayoun Safarpour, and it first shipped in 2026. The project is developed in the open on GitHub with 13 commits in the last 90 days. Among its 4 catalogued features are Drift Detection, Anchor Set Analysis, and LLM Judge Monitoring.
Summary written by a language model from the project’s public pages.
- Drift Detection
- Anchor Set Analysis
- LLM Judge Monitoring
- Evaluation Stability
Tags
llm-eval-driftjudge-drift-detectionanchor-based-evaluationllm-as-judgeeval-observability
AI capabilities
Structured
Inference: Cloud API
Built with & integrations
Runs on
CLIAPI-only
Trust & compliance
License
MIT
Verified signals
Indexing history
1What PulseGate has recorded for this listing
- Indexed5 Aug · 09:37 UTCjudge-drift-sentinel verified against its public sourceSource: PulseGate · Open
Frequently asked questions about judge-drift-sentinel
- What is judge-drift-sentinel?
- Judge-drift-sentinel focuses on detecting whether shifts in LLM evaluation scores are caused by changes to the system under test or by drift in the LLM judge itself. It is catalogued under LLM eval & observability on PulseGate.
- Who should use judge-drift-sentinel?
- judge-drift-sentinel is an open-source project built for AI engineers and LLM application developers.
- Is judge-drift-sentinel free?
- Yes — judge-drift-sentinel is open source under the MIT license and free to use.
- What platforms does judge-drift-sentinel run on?
- judge-drift-sentinel runs on the command line and API.
- Is judge-drift-sentinel still maintained?
- The GitHub repository shows 13 commits in the last 90 days.
- What projects are similar to judge-drift-sentinel?
- Similar projects tracked by PulseGate include constraintloop, trajectory-judge, and DeFake.constraintlooptrajectory-judgeDeFake
- Who develops judge-drift-sentinel?
- judge-drift-sentinel is developed by Homayoun Safarpour.
- When did judge-drift-sentinel launch?
- judge-drift-sentinel first shipped in 2026.
Also in LLM eval & observability
Same category — not a similarity match