vectory
vectory is an open-source AI evaluation toolkit designed for trace-aware assessment of LLMs, agents, and RAG systems. It includes a local Vectory Benchmark runner and a Streamlit-based interface. Available as a Python package under the MIT license, it helps machine learning engineers and developers rigorously test and observe their AI applications.
Open SourceMITWebCLISelf-hosted
1star
4features
2026since
Overview
4 featuresvectory sits in PulseGate's LLM eval & observability category. It focuses on evaluating and benchmarking LLM and agent performance with traceable, local tools instead of opaque cloud evaluators. vectory is an open-source project aimed at developers. vectory is open source under the MIT license. vectory is available on the web and the command line, and it can be self-hosted.
Behind vectory is tacticaledge, and it first shipped in 2026. Development happens publicly on GitHub with 23 commits in the last 90 days. Key capabilities include AI Evaluation, Benchmark Runner, and Trace Awareness.
Summary written by a language model from the project’s public pages.
- AI Evaluation
- Benchmark Runner
- Trace Awareness
- RAG Evaluation
Tags
ai-evaluationllm-benchmarkingrag-evaltrace-aware
AI capabilities
Text
Inference: Local
Built with & integrations
AI providers
anthropicopenai
Runs on
BrowserCLISelf-hosted
Trust & compliance
License
MIT
Verified signals
Indexing history
1What PulseGate has recorded for this listing
Frequently asked questions about vectory
- What is vectory?
- Vectory focuses on evaluating and benchmarking LLM and agent performance with traceable, local tools instead of opaque cloud evaluators. It is catalogued under LLM eval & observability on PulseGate.
- Who should use vectory?
- vectory is an open-source project built for developers.
- Does vectory have a free plan?
- Yes — vectory is open source under the MIT license and free to use.
- What platforms does vectory run on?
- vectory runs on the web and the command line. It can also be self-hosted.
- Is vectory still maintained?
- The GitHub repository shows 23 commits in the last 90 days.
- Who makes vectory?
- vectory is developed by tacticaledge.
- How long has vectory been around?
- vectory first shipped in 2026.
- Is vectory open source?
- Yes — vectory is open source under the MIT license, developed on GitHub.
Similar apps
Closest matches by what these projects do