whatifd is an open-source CLI tool for running experiments on LLM behavior changes. It allows users to fork production traces, replay with proposed changes, score differences, and generate PR-ready verdict reports for AI model evaluation.
In the LLM eval & observability space, whatifd takes a focused approach. It helps developers evaluate and compare the impact of changes on LLM behavior using production traces. It is built as an open-source project for AI engineers. whatifd is open source under the Apache-2.0 license. whatifd is available on the web and the command line.
whatifd first shipped in 2026. PulseGate's similarity index finds few close equivalents — whatifd occupies a relatively distinct niche. Key capabilities include experiment runner, LLM behavior analysis, and trace replay.
Latest indexed changes and source events
Other apps tracked under the same category.
