Who should use agentsec-eval?

agentsec-eval is an open-source project built for AI security researchers and developers.

Is agentsec-eval free?

Yes — agentsec-eval is open source under the MIT license and free to use.

What platforms does agentsec-eval run on?

agentsec-eval runs on the command line. It can also be self-hosted.

Is agentsec-eval still maintained?

Activity appears to have slowed — PulseGate's most recent liveness signals classify this listing as stale.

What projects are similar to agentsec-eval?

Similar projects tracked by PulseGate include agent-eval, agentsec-cli, and openagent-eval.

Who makes agentsec-eval?

agentsec-eval is developed by raoliaoyuan.

How long has agentsec-eval been around?

agentsec-eval first shipped in 2026.

Back to the index

agentsec-eval

Name: agentsec-eval
Author: raoliaoyuan

github.comInfrastructure

agentsec-eval is an open-source CLI framework for evaluating the security of AI agents. It provides adversarial test runners, server-side audits, and scoring mechanisms to help researchers and developers identify vulnerabilities such as prompt injection and improve agent robustness.

Open SourceMITCLISelf-hosted

Visit github.com

1alternative

5features

2026since

Overview

5 features

In the LLM eval & observability space, agentsec-eval takes a focused approach. It focuses on assessing and improving the security of AI agents through adversarial testing and audits. It is built as an open-source project for AI security researchers and developers. The project is open source (MIT). It runs on the command line, and it can be self-hosted.

It is developed by raoliaoyuan, and it first shipped in 2026. Key capabilities include adversarial testing, security audit, and scoring system.

Summary written by a language model from the project’s public pages.

Adversarial testing
Security audit
Scoring system
Server-side audit
Prompt injection testing

Built with & integrations

Runs on

CLISelf-hosted

Trust & compliance

License

MIT

Verified signals

HTTPSOpen SourceFree tierGitHub

Indexing history

What PulseGate has recorded for this listing

Indexed27 Jun · 08:14 UTC
Listing verified against its public source
Source: PulseGate · Open

Frequently asked questions about agentsec-eval

What does agentsec-eval do?: Agentsec-eval focuses on assessing and improving the security of AI agents through adversarial testing and audits. It is catalogued under LLM eval & observability on PulseGate.
Who should use agentsec-eval?: agentsec-eval is an open-source project built for AI security researchers and developers.
Is agentsec-eval free?: Yes — agentsec-eval is open source under the MIT license and free to use.
What platforms does agentsec-eval run on?: agentsec-eval runs on the command line. It can also be self-hosted.
Is agentsec-eval still maintained?: Activity appears to have slowed — PulseGate's most recent liveness signals classify this listing as stale.
What projects are similar to agentsec-eval?: Similar projects tracked by PulseGate include agent-eval, agentsec-cli, and openagent-eval.agent-eval agentsec-cli openagent-eval
Who makes agentsec-eval?: agentsec-eval is developed by raoliaoyuan.
How long has agentsec-eval been around?: agentsec-eval first shipped in 2026.

At a glance

Platforms: Cli
Languages: English
Open source: Yes (GitHub)
License: MIT
Activity: 🟡 Stale
Built for: AI security researchers and developers
Model: Open source
Solves: Assessing and improving the security of AI agents through adversarial testing and audits.

Registered as

GitHub: raoliaoyuan/AgentSec
PyPI: agentsec-eval

Developer

raoliaoyuan

GitHub

Open source

View on GitHub

Index record

Identity confidence: Low · 64
Indexed: 27 Jun 2026
Lifecycle: Dead
Activity: Stale
Last seen: 27 Jun 2026

Identity audit (12)

Slug: agentsec-eval-pypi-org
Lifecycle last checked: 17 Aug 2026
Verification state: Indexed for public listing
Listing state: Listed: yes
Index status: Included in index
Latest evidence snapshot: 27 Jun 2026
Timeline basis: Indexed-at chronology. This listing's first-seen date was written by the catalog backfill, not observed here, so it is not treated as a sighting.
Name from: Written by a language model from the project's public pages.
Category from: Assigned by a language model.
Summary from: Written by a language model from public pages.
Languages from: Detected by a language model from page content.
Canonical URL: https://github.com/raoliaoyuan/AgentSec

Ship this? Send a correction — no account, and you get a link to follow it.

Similar projects

Closest matches by what these projects do