Kensa is an open-source tool designed to detect and prevent regressions in agent behavior by transforming agent traces into pytest-native evaluation tests. Its primary use is for teams seeking to ensure that changes in coding agent behavior are caught during continuous integration (CI), rather than going unnoticed. By leveraging real traces from agent runs, Kensa creates tests that reflect actual behavior, so that regressions—such as unintended changes in tool use or decision-making—cause CI to fail and block merges.
The workflow begins with importing traces, which can be sourced from platforms like Langfuse, exported files in JSON or JSONL format, or captured locally using Kensa’s instrumentation functions. Kensa then analyzes these traces to generate reviewable behavior candidates as plain pytest files, or users can write their own tests. The tool first applies deterministic and trace-based checks, resorting to an LLM-based judge only when necessary. Each run produces traces that can be mined for more precise regression tests. Kensa’s approach differs from traditional observability dashboards or prompt playgrounds by focusing on trace-backed evaluations rather than one-off experiments or generic metrics.
Kensa integrates directly into existing Python repositories by adding an evaluation layer on top of pytest, requiring minimal changes to agent code. Instrumentation is accomplished with a single call at startup and optional wrapping of model calls. The kensa_run fixture can drive agents in any language, either via subprocess or HTTP, and traces can be imported from any runtime. The platform also provides primitives such as kensa_run, kensa_trace, judge(), and trials to rerun nondeterministic cases. Sensitive trace data is handled with redaction on import, and additional strict redaction is available via an extra package using a spaCy model.
0 licensed, making it free to use except for any costs associated with optional LLM judge calls. Kensa is positioned as a regression testing solution for agent behavior, enabling repository-native gates that tie pass/fail criteria to the agent’s real-world job, and is intended for teams who want to maintain robust and reliable agent deployments.
Kensa is a LLM eval & observability product. It focuses on automating the creation and execution of evals for coding agents to streamline testing and validation. It is built as an open-source project for developers building or testing coding agents. Kensa is open source under the MIT license. It runs on the command line.
Kensa builds and maintains Kensa, and the product first shipped in 2026. Development happens publicly on GitHub with 216 commits in the last 90 days. Key capabilities include eval generation, scenario tracing, and LLM call capture.
Latest indexed changes and source events
Other apps tracked under the same category.
