SkillScan

SkillScan was a free, local, and private scanner designed to detect security issues in AI agent skill files. The tool operated without requiring a GPU, incurred no per-scan fees, and did not require users to send their files to a company, emphasizing privacy and local analysis. Its approach involved static analysis and the use of pattern rules or a lightweight classifier to identify potentially malicious behaviors or instructions within skill files.

The scanner was developed as an open-source project, with its code remaining publicly available even after its retirement. SkillScan's methodology centered on identifying problematic patterns, but its creator found that such tools, including SkillScan itself, often missed a significant portion of novel attacks or over-blocked to compensate. The tool was benchmarked alongside other similar scanners, and results indicated that static or rule-based approaches detected only a fraction of sophisticated threats, particularly those involving intent or context that simple pattern matching could not discern.

SkillScan's development and subsequent benchmarking were aimed at providing detection capabilities that users could fully control—local, private, and open-source—without reliance on external services or cloud-based models. However, the project's findings highlighted the limitations of static and rule-based detection in this domain, especially when compared to more advanced reasoning models. The scanner has since been retired, but its code remains accessible for reference.