llm-canary is an open-source regression testing tool designed for developers building LLM-powered applications. It enables YAML-based test suites, offline-first assertions, baseline diffing, and agent-trace policy gates, making it easy to automate evaluation and catch regressions in CI pipelines. Ideal for teams seeking robust LLM app quality assurance.
llm-canary sits in PulseGate's LLM eval & observability category. It focuses on ensuring LLM app reliability by automating regression testing and baseline comparisons in CI workflows. It is built as an open-source project for LLM app developers. llm-canary is open source under the MIT license. llm-canary is available on the command line.
It is developed by okssusucha, and the product first shipped in 2026. Development happens publicly on GitHub with 8 commits in the last 90 days. Key capabilities include YAML test suites, offline assertions, and baseline diffing.
Latest indexed changes and source events
Other apps tracked under the same category.
