inspect-ai
inspect-ai is an open-source framework designed to automate and standardize the evaluation of large language models. It provides tools for running automated tests, defining custom metrics, and integrating with Python workflows. Ideal for AI researchers and developers seeking robust LLM evaluation pipelines.
Overview
5 featuresinspect-ai sits in PulseGate's LLM eval & observability category. It focuses on automating and standardizing the evaluation of large language models for developers and researchers. It is built as an open-source project for AI researchers and developers. inspect-ai is open source under the MIT license. It ships for the web and the command line, and it can be self-hosted.
It is developed by UK Government Department for Business, Energy & Industrial Strategy (United Kingdom), and it first shipped in 2024. Development happens publicly on GitHub with 2.3k stars and 1.2k commits in the last 90 days. Among its 5 catalogued features are evaluation framework, automated testing, and LLM support.
Summary written by a language model from the project’s public pages.
- Evaluation framework
- Automated testing
- LLM support
- Custom metrics
- Python integration
Built with & integrations
Trust & compliance
Indexing history
1What PulseGate has recorded for this listing
Frequently asked questions about inspect-ai
- What does inspect-ai do?
- Inspect-ai focuses on automating and standardizing the evaluation of large language models for developers and researchers. It is catalogued under LLM eval & observability on PulseGate.
- Who is inspect-ai for?
- inspect-ai is an open-source project built for AI researchers and developers.
- Does inspect-ai have a free plan?
- Yes — inspect-ai is open source under the MIT license and free to use.
- What platforms does inspect-ai run on?
- inspect-ai runs on the web and the command line. It can also be self-hosted.
- Is inspect-ai still maintained?
- The GitHub repository shows 1.2k commits in the last 90 days.
- Who develops inspect-ai?
- inspect-ai is developed by UK Government Department for Business, Energy & Industrial Strategy, based in the United Kingdom.
- When did inspect-ai launch?
- inspect-ai first shipped in 2024.
- Is inspect-ai open source?
- Yes — inspect-ai is open source under the MIT license, developed on GitHub.
Similar projects
Closest matches by what these projects do