judgegate is an open-source CLI tool for evaluating LLM judges in continuous integration workflows. It provides agreement statistics against human labels, bias probes, and a label budget calculator, supporting robust AI evaluation and monitoring for developers and researchers.
judgegate sits in PulseGate's LLM eval & observability category. It helps developers evaluate and monitor the reliability and bias of LLM judges in CI pipelines. judgegate is an open-source project aimed at AI evaluation engineers. The project is open source (MIT). The product ships for the command line, and it can be self-hosted.
yashchimata builds and maintains judgegate, and the product first shipped in 2026. The project is developed in the open on GitHub with 1 commits in the last 90 days. Among its 5 catalogued features are agreement statistics, bias probes, and label budget calculator.
Latest indexed changes and source events
Other apps tracked under the same category.
