Skip to content
Back to the index

FreshBench

PyPIInfrastructure

PulseGate's liveness check found it on 14 Sep 2026; it is registered on GitHub and PyPI and has been in the index since 21 Jul 2026. How this is checked

FreshBench is a CLI tool and Python package that runs fresh, local-first benchmarks for OpenAI-compatible language models, including support for llama.cpp. It enables developers to evaluate LLM performance on their own hardware without depending on external services. The project is open source under Apache-2.0 and hosted on GitHub.

Inferred · not functionally tested

Open SourceApache-2.0CLI
Visit PyPI

Overview

4 features

Purpose: Evaluating and benchmarking local and OpenAI-compatible LLMs without relying on centralized services.

Inferred · not functionally tested

Audience: developers

Inferred · not functionally tested

Functions: analytics

Inferred · not functionally tested

Interfaces: API: unknown · MCP: unknown · CLI: indicated (inferred, not tested) · Self-hosting: unknown

Recorded constraints: pricing: open_source · license: Apache-2.0 · platforms: CLI · deployment: cli

Constraint provenance is unknown; confirm requirements with the publisher.

Record sources: pypi.org · github.com. These links do not verify the individual claims.

FreshBench is a LLM evaluation & benchmarks project. Inferred · not functionally tested: It focuses on evaluating and benchmarking local and OpenAI-compatible LLMs without relying on centralized services. Inferred · not functionally tested: FreshBench is an open-source project aimed at developers. Basis unknown · not verified: FreshBench is open source under the Apache-2.0 license. Basis unknown · not verified: FreshBench is available on the command line.

It is developed by DogukanUrker, and it first shipped in 2026. The project is developed in the open on GitHub with 4 commits in the last 90 days. Inferred · not functionally tested: Among its 4 catalogued features are Local Benchmarks, CLI Interface, and OpenAI Compatibility.

Summary written by a language model from the project’s public pages.

Tasks: Inferred · not functionally tested

  • Local Benchmarks
  • CLI Interface
  • OpenAI Compatibility
  • LLM Evaluation

Topics: Inferred · not functionally tested

Tags
llm-benchmarkinglocal-evaluationopenai-compatiblellama-cpp
AI capabilities
Text
Inference: Local

JSON profile · Text profile · Access guide

Built with & integrations

AI providers
multiple
Runs on
CLI

Trust & compliance

License
Apache-2.0
Public signals
HTTPSOpen SourceFree tierGitHubActive maintenance

Indexing history

1

What PulseGate has recorded for this listing

  1. Indexed21 Jul · 16:49 UTC
    freshbench seen via PyPI Bulk Enumerator
    Source: PyPI Bulk Enumerator · Open

Frequently asked questions about FreshBench

What is FreshBench?
Inferred · not functionally tested: FreshBench focuses on evaluating and benchmarking local and OpenAI-compatible LLMs without relying on centralized services. It is catalogued under LLM evaluation & benchmarks on PulseGate.
Who should use FreshBench?
Inferred · not functionally tested: FreshBench is an open-source project built for developers.
Is FreshBench free?
Basis unknown · not verified: Yes — FreshBench is open source under the Apache-2.0 license and free to use.
What platforms does FreshBench run on?
Basis unknown · not verified: FreshBench runs on the command line.
Is FreshBench still active?
PulseGate's liveness check found it on 14 Sep 2026. Its GitHub repository shows 4 commits in the last 90 days.
Who makes FreshBench?
FreshBench is developed by DogukanUrker.
When did FreshBench launch?
FreshBench first shipped in 2026.
Is FreshBench open source?
Basis unknown · not verified: Yes — FreshBench is open source under the Apache-2.0 license, developed on GitHub.

Similar projects

Closest matches by what these projects do