evalshift
evalshift is an open-source CLI tool that allows users to run prompts on two different large language models and statistically determine if there are regressions in output quality. It is designed for machine learning engineers and teams evaluating LLM migrations or updates.
Overview
5 featuresevalshift is a LLM evaluation & benchmarks project. It focuses on detecting regressions in LLM outputs when migrating or comparing models. evalshift is an open-source project aimed at machine learning engineers. evalshift is open source under the AGPL-3.0-or-later license. evalshift is available on the command line.
evalshift first shipped in 2026. Development happens publicly on GitHub with 17 commits in the last 90 days. Key capabilities include prompt comparison, statistical analysis, and regression detection.
Summary written by a language model from the project’s public pages.
- Prompt comparison
- Statistical analysis
- Regression detection
- LLM migration support
- Command-line interface
Built with & integrations
Trust & compliance
Indexing history
1What PulseGate has recorded for this listing
Frequently asked questions about evalshift
- What does evalshift do?
- Evalshift focuses on detecting regressions in LLM outputs when migrating or comparing models. It is catalogued under LLM evaluation & benchmarks on PulseGate.
- Who should use evalshift?
- evalshift is an open-source project built for machine learning engineers.
- Is evalshift free?
- Yes — evalshift is open source under the AGPL-3.0-or-later license and free to use.
- What platforms does evalshift run on?
- evalshift runs on the command line.
- Is evalshift still maintained?
- The GitHub repository shows 17 commits in the last 90 days.
- How long has evalshift been around?
- evalshift first shipped in 2026.
- Is evalshift open source?
- Yes — evalshift is open source under the AGPL-3.0-or-later license, developed on GitHub.
Similar projects
Closest matches by what these projects do