FreshBench is a CLI tool and Python package that runs fresh, local-first benchmarks for OpenAI-compatible language models, including support for llama.cpp. It enables developers to evaluate LLM performance on their own hardware without depending on external services. The project is open source under Apache-2.0 and hosted on GitHub.
In the LLM eval & observability space, FreshBench takes a focused approach. It focuses on evaluating and benchmarking local and OpenAI-compatible LLMs without relying on centralized services. FreshBench is an open-source project aimed at developers. The project is open source (Apache-2.0). FreshBench is available on the command line.
DogukanUrker builds and maintains FreshBench, and the product first shipped in 2026. The project is developed in the open on GitHub with 4 commits in the last 90 days. Among its 4 catalogued features are Local Benchmarks, CLI Interface, and OpenAI Compatibility.
Latest indexed changes and source events
Other apps tracked under the same category.
