capevalkit
capevalkit is an open-source CLI toolkit for reproducible evaluation of captions generated by vision-language models. It supports per-metric environments, enabling researchers and engineers to benchmark and compare VLM outputs with consistent, reliable metrics.
Open SourceBSD-3-Clause-ClearCLI
7stars
5features
2026since
Overview
5 featurescapevalkit is a LLM eval & observability project. It focuses on ensuring reproducible and reliable evaluation of captions generated by vision-language models. It is built as an open-source project for AI researchers and ML engineers. capevalkit is open source under the BSD-3-Clause-Clear license. It ships for the command line.
capevalkit first shipped in 2026. Development happens publicly on GitHub with 32 commits in the last 90 days. Among its 5 catalogued features are caption evaluation, metric reproducibility, and VLM support.
Summary written by a language model from the project’s public pages.
- Caption evaluation
- Metric reproducibility
- VLM support
- CLI interface
- Custom environments
Tags
caption-evaluationvlm-metricsreproducibility
AI capabilities
TextImage
Weights: Open
Built with & integrations
Runs on
CLI
Trust & compliance
License
BSD-3-Clause-Clear
Verified signals
Indexing history
1What PulseGate has recorded for this listing
Frequently asked questions about capevalkit
- What is capevalkit?
- Capevalkit focuses on ensuring reproducible and reliable evaluation of captions generated by vision-language models. It is catalogued under LLM eval & observability on PulseGate.
- Who should use capevalkit?
- capevalkit is an open-source project built for AI researchers and ML engineers.
- Is capevalkit free?
- Yes — capevalkit is open source under the BSD-3-Clause-Clear license and free to use.
- What platforms does capevalkit run on?
- capevalkit runs on the command line.
- Is capevalkit still active?
- The GitHub repository shows 32 commits in the last 90 days.
- What are alternatives to capevalkit?
- Similar projects tracked by PulseGate include constraintloop, trajectory-judge, and DeFake.constraintlooptrajectory-judgeDeFake
- How long has capevalkit been around?
- capevalkit first shipped in 2026.
- Is capevalkit open source?
- Yes — capevalkit is open source under the BSD-3-Clause-Clear license, developed on GitHub.
Also in LLM eval & observability
Same category — not a similarity match