Hotato

Hotato is an open-source tool designed for analyzing and evaluating the performance of production voice agents, with a focus on turn-taking dynamics during calls. It serves as a 'flight recorder' for voice agent interactions, enabling users to identify and investigate moments where the agent's behavior may not align with conversational expectations, such as talking over a caller or failing to yield the floor at appropriate times.

The tool operates by scanning call histories—either from bundled demo calls or a user's own recordings—to surface and rank candidate moments of interest. These moments are categorized by events like talk-over durations or missed interruptions, and each candidate is accompanied by an inline audio clip for review. Users can label these moments (for example, as 'yield' or 'hold'), and confirmed issues can be promoted to fixtures. These fixtures become permanent tests that are checked on every code push, supporting continuous integration workflows by ensuring that specific conversational failures are not reintroduced after a fix.

Hotato runs locally and does not require user accounts, API keys, or network connectivity, ensuring that call data remains private and on the user's machine. It supports analysis of stereo WAV recordings and can be integrated with various call stacks. The tool provides outputs such as shareable candidate cards, HTML reports with embedded audio, and portable contracts for regression testing. Reports aggregate and visualize metrics like time-to-yield and talk-over histograms directly from the recorded data.

While solo developers can use Hotato independently, there are additional features aimed at teams managing multiple agents, including an optional self-hosted control plane. The open-source nature of Hotato allows users to inspect, modify, and extend its functionality as needed for their own voice agent evaluation workflows.