traceval compiles production LLM-agent traces from tools like OTel, Langfuse, and LangSmith into runnable pytest evaluation suites. It supports regression detection and judge calibration, helping AI engineers automate and improve LLM agent evaluation workflows.
In the LLM eval & observability space, traceval takes a focused approach. It focuses on automating the evaluation and regression testing of LLM-agent traces for reliable model performance. traceval is an open-source project aimed at AI engineers and developers evaluating LLM agents. The project is open source (MIT). traceval is available on the command line, and it can be self-hosted.
Behind traceval is theramkm, and the product first shipped in 2026. The project is developed in the open on GitHub with 39 commits in the last 90 days. Across PulseGate's embedding index, traceval has few near neighbours, marking it as relatively distinct. Among its 5 catalogued features are trace compilation, pytest integration, and regression detection.
Latest indexed changes and source events
Other apps tracked under the same category.
