Free and Open Source AI Security Tools

Last Updated on June 15, 2026 by Arnav Sharma

The attack surface for AI systems has expanded faster than the tooling available to defend it. Security professionals deploying large language models, agentic AI workflows, and generative AI pipelines face a category of vulnerability that existing security tools were not designed to address. Prompt injection, model manipulation, supply chain compromise, and autonomous agent goal hijacking require purpose-built approaches. Free and open-source AI security tools have matured significantly in 2025 and 2026, giving security architects, red teams, and developers and security teams access to production-grade capabilities without commercial licensing costs.

This guide covers eight open-source tools that are actively maintained and seeing real-world adoption in 2026. Each tool is mapped to its deployment phase, OWASP coverage, and suitability for agentic AI workloads. It also addresses the supply chain trust questions that the March 2026 Trivy compromise raised for anyone consuming open-source tools in a security context.

Why Free and Open Source AI Security Tools Matter in 2026

Three structural forces are converging to make open source AI security tooling urgent rather than optional.

The first is AI adoption outpacing security investment. According to Cycode’s 2026 State of Product Security report, every organisation surveyed had AI-generated code in their codebase, while 81 per cent lacked visibility into AI usage across their software development lifecycle. Wiz’s State of AI in the Cloud 2026 found that approximately one in five organisations using AI-powered development platforms had applications affected by widespread security flaws. When AI-generated defaults are repeated at scale, insecure patterns become embedded across systems rather than remaining isolated issues. Security challenges that were once edge cases are now default conditions.

The second force is regulatory timelines. The European Union AI Act’s high-risk AI obligations come into effect in August 2026. The Colorado AI Act becomes enforceable in June 2026. The Australian Government’s AI frameworks, including the ACSC’s AI security guidance and the proposed mandatory guardrails for high-risk AI, are tightening the compliance requirements for AI systems in regulated industries. Open-source tools that generate auditable outputs, structured reports, and integration logs become essential evidence for demonstrating due diligence.

The third force is framework maturity. OWASP published the Top 10 for Large Language Model Applications in its 2025 edition and followed it with the OWASP Top 10 for Agentic Applications (ASI) in December 2025, the first formal taxonomy of risks specific to autonomous AI agents. These frameworks give security teams a structured vocabulary for threat modelling that maps directly to tooling. Security teams can now buy down specific risks with specific ai tools rather than relying on general-purpose security solutions that were never designed with AI systems in mind.

Understanding the AI Security Tool Landscape

Before selecting tools, security teams need to understand where in the AI system lifecycle security vulnerabilities exist and where tooling intervenes. Three deployment phases define the structure:

Pre-deployment testing covers vulnerability scanning, red teaming, and adversarial probing of AI models and applications before they reach production. The goal is to identify exploitable behaviours, prompt injection vectors, data leakage paths, and insecure output handling during development and CI/CD workflows. This phase often involves analysing source code for AI-generated code patterns that introduce insecure defaults.

Runtime protection covers guardrails, input/output scanning, and policy enforcement for AI systems operating in production. The goal is to detect and block adversarial inputs, enforce behavioural constraints on AI agents, and prevent sensitive data from appearing in model outputs. Automated security scanning at this layer is essential for cybersecurity AI deployments where human review of every interaction is not feasible.

Supply chain and governance covers model provenance, AI bill of materials (AI-BOM) tracking, dependency scanning for AI packages, and policy governance for agentic workflows. The goal is to ensure the integrity of the models, packages, and tools that compose the AI system before and during operation. Security posture management for AI assets starts here.

Each tool category in this guide maps to one or more of these phases. Security teams should select tooling across all three phases rather than relying on a single layer of defence.

Free and Open Source AI Security Tools, Mapped by Use Case

The following eight tools represent the highest-value open source options available in 2026. Selection criteria include active maintenance, community adoption, OWASP alignment, and suitability for enterprise security teams rather than research use only.

Garak: LLM Vulnerability Scanner

Garak (Generative AI Red-teaming and Assessment Kit) is NVIDIA’s open source LLM vulnerability scanner, released under the Apache 2.0 licence. It functions as the LLM equivalent of a network vulnerability scanner: point it at a model endpoint, configure probe modules, and it systematically generates adversarial inputs, sends them to the model, and reports on which vulnerability classes the model exhibits. AI model security research teams at NVIDIA, Fujitsu, Microsoft, and Cisco have all recommended Garak as a standard AI testing tool.

The tool carries 7,500 or more GitHub stars as of mid-2026 and is actively maintained with regular releases. Version 0.15.0, shipped in May 2026, introduced multi-turn GOAT probes and an Agent-breaker probe specifically designed to test tool-use behaviours available to LLM agents, making it one of the first open-source tools to directly address agentic attack surfaces at the AI agent security boundary.

Garak’s probe taxonomy maps directly to the OWASP LLM Top 10 for 2025. Prompt injection probes map to LLM01 (Prompt Injection). Leakreplay and training data probes map to LLM02 (Sensitive Information Disclosure). Package hallucination probes map to LLM03 (Supply Chain). The tool reports results in JSONL and HTML formats suitable for CI/CD integration. Because LLM output is non-deterministic, Garak runs each probe ten times by default, capturing models that pass a jailbreak nine times but fail once: a 10 per cent bypass rate is a documented finding.

Garak is a pre-deployment tool and a red teaming ai tool. It is not a runtime guardrail. Security teams should run Garak scans against model endpoints during development, on version upgrades, and as part of release gate testing.

Primary use: Pre-deployment LLM vulnerability scanning, AI red teaming Licence: Apache 2.0 Key OWASP coverage: LLM01, LLM02, LLM03, LLM06 Agentic support: Yes (Agent-breaker probe in v0.15.0) CI/CD integration: Yes, via CLI and JSON config

Promptfoo: CI/CD-Integrated LLM Testing

Promptfoo is an MIT-licensed LLM evaluation and security testing framework used by over 300,000 developers and integrated into CI/CD pipelines at a substantial proportion of Fortune 500 organisations. The tool treats prompts as testable code: you define test cases with expected behaviours in YAML, run them against any LLM provider, and receive structured reports showing what passed and what failed. It is one of the most widely adopted dedicated AI security testing tools available to developers and security professionals today.

Its red teaming module automatically generates adversarial inputs targeting 50 or more vulnerability types, including prompt injection, jailbreaks, PII leakage, tool exploit attempts, and policy-violating outputs. The framework integrates natively with GitHub Actions, allowing security teams to block deployments when vulnerability scores exceed defined thresholds, exactly like a software security test suite. It supports AI services from OpenAI, Anthropic, Azure AI Foundry, Google, AWS Bedrock, and custom endpoints, making it suitable for multi-cloud AI environments.

In March 2026, OpenAI acquired Promptfoo with the stated intention of integrating the technology into OpenAI Frontier, its enterprise agent platform. This acquisition has implications for security teams evaluating tool independence. The core MIT-licensed codebase remains open source and the tool continues to function with non-OpenAI providers. However, security architects evaluating tool neutrality for regulated environments or multi-vendor AI strategies should note this context when assessing long-term dependency on the platform.

For agentic AI security, Promptfoo’s ability to test tool-use behaviours, simulate indirect prompt injection via tool outputs, and evaluate multi-turn conversation security makes it one of the most practically useful offensive security frameworks available in the open source space.

Primary use: CI/CD-integrated LLM testing, pre-deployment red teaming Licence: MIT Key OWASP coverage:LLM01, LLM02, LLM06, LLM07, LLM08 Agentic support: Yes (tool-use testing, multi-turn simulation) CI/CD integration: Native GitHub Actions support

DeepTeam: Agentic AI Red Teaming

DeepTeam is an Apache 2.0 open source red teaming framework from Confident AI, purpose-built for LLM systems and agentic AI applications. It represents an ai-first security approach to agentic security: where Garak and Promptfoo provide broad LLM vulnerability coverage, DeepTeam’s architecture is specifically designed for the threat model that emerges when AI agents can access tools, maintain memory across sessions, and take autonomous actions.

The framework simulates jailbreaking, prompt injection, multi-turn exploitation, memory poisoning, and tool misuse attacks. It exposes 50 or more pre-built vulnerability types and includes production-ready guardrails for real-time binary classification of inputs and outputs. Seven guards are available out of the box: ToxicityGuard, PromptInjectionGuard, PrivacyGuard, IllegalGuard, HallucinationGuard, TopicalGuard, and CybersecurityGuard. These can be deployed as lightweight classifiers in front of production agentic workflows.

DeepTeam is the most relevant open source tool for teams implementing the OWASP Top 10 for Agentic Applications (ASI) framework published in December 2025. The ASI Top 10 addresses risks including goal hijacking, tool misuse, identity abuse, memory poisoning, and cascading failures in multi-agent systems. DeepTeam’s attack simulation capabilities map directly to these risk categories, providing the most comprehensive AI security coverage for agentic workloads available from any open source project.

The tool runs locally, integrates with Python workflows, and supports YAML-based CLI configuration. For teams building on LangChain, LangGraph, or custom agentic frameworks, DeepTeam provides the most directly applicable open source red teaming coverage available. It is also one of the most useful tools like Garak that explicitly addresses autonomous AI agents rather than static LLM endpoints.

Primary use: Agentic AI red teaming, runtime guardrails for agents Licence: Apache 2.0 Key OWASP coverage:OWASP ASI (Agentic Top 10), LLM01, LLM02, LLM08 Agentic support: Purpose-built for agentic AI workloadsCI/CD integration: Yes, via CLI and Python API

IBM Adversarial Robustness Toolbox (ART)

The Adversarial Robustness Toolbox is an open-source AI and machine learning security library developed by IBM Research and now maintained under the Linux Foundation AI and Data Foundation. It is the most comprehensive framework available for adversarial machine learning testing, covering four attack categories: evasion, poisoning, extraction, and inference. Security researchers at IBM, academic institutions, and enterprise security teams use ART as the reference implementation for formal adversarial AI assessments.

ART is distinct from the LLM-focused tools above in that it addresses the full machine learning model lifecycle, not just language model behaviours. These are specialized AI attack vectors that require dedicated testing tooling beyond standard prompt-based red teaming. Evasion attacks test whether an adversary can craft inputs that fool a model into misclassification. Poisoning attacks simulate training data manipulation that degrades model behaviour. Extraction attacks assess whether model functionality can be replicated by querying it systematically. Membership inference attacks test whether training data membership can be inferred from model outputs.

The library integrates with TensorFlow, PyTorch, Keras, scikit-learn, and XGBoost, making it applicable across computer vision, NLP, and tabular prediction systems. For security teams responsible for models beyond large language models, such as fraud detection, anomaly detection, or image classification systems, ART provides the adversarial testing coverage that LLM-specific tools do not address.

For large language model applications specifically, ART’s text-based adversarial attack modules and its alignment with MITRE ATLAS attack taxonomy make it the appropriate tool for teams conducting formal adversarial AI security assessments.

Primary use: Adversarial ML testing, model robustness evaluation Licence: MIT Key OWASP coverage: LLM03, LLM04 (denial of service), LLM09 (misinformation) MITRE ATLAS alignment: Yes CI/CD integration: Via Python scripting

NVIDIA NeMo Guardrails

NeMo Guardrails is an open source Python toolkit for adding programmable guardrails to LLM-based applications. Unlike the red teaming tools above, which probe for vulnerabilities, NeMo Guardrails is a runtime protection layer deployed alongside production LLM applications to enforce behavioural constraints on inputs, dialogue flows, retrieval operations, and outputs. It is one of the most widely adopted ai-powered security tools for production LLM deployment.

The toolkit uses a Colang/YAML configuration approach that allows developers to define conversation rails without modifying the underlying model. Input rails filter incoming user messages. Dialogue rails enforce topic and intent boundaries. Retrieval rails govern what content can be injected into model context. Output rails block or rewrite responses that violate defined policies. This layered structure gives security teams and application teams shared control over model behaviour in production, contributing to overall security posture management for LLM-based applications.

NeMo Guardrails integrates directly with Garak for vulnerability assessment: security teams can run Garak probe suites against NeMo-protected applications to measure the reduction in attack success rates that guardrails provide. Research published alongside the toolkit demonstrated measurable reduction in jailbreak and prompt injection success rates when input and dialogue rails were combined. This integration enables automated security validation of guardrail effectiveness as part of a release workflow.

For agentic AI applications, NeMo’s execution rails can restrict which tools an agent is permitted to call and under what conditions, providing a mechanism for enforcing least-privilege principles on autonomous AI agents. This capability aligns directly with the ASI03 (Unsafe Agent Actions) risk in the OWASP Agentic Top 10.

Primary use: Runtime LLM guardrails, production agentic AI protection Licence: Apache 2.0 Key OWASP coverage: LLM01 (Prompt Injection runtime), LLM06, LLM08 Agentic support: Yes (execution rails for tool use governance) Integration: Garak, LangChain, custom LLM frameworks

Microsoft Agent Governance Toolkit

Released in April 2026 under the MIT licence by Microsoft’s open source team, the Agent Governance Toolkit is the first open-source framework specifically designed to address all ten risks in the OWASP Top 10 for Agentic Applications. The toolkit provides runtime security governance for autonomous AI agents, with deterministic sub-millisecond policy enforcement that does not require LLM inference to make security decisions. Developers and security teams building on Model Context Protocol can use it to add a dedicated AI security governance layer without rewriting existing agent logic.

The architecture centres on a policy engine, a capability sandboxing layer, and a Model Context Protocol (MCP) security gateway that intercepts tool calls before they execute. Each component maps to specific OWASP ASI risks. Goal hijacking (ASI01) is addressed by a semantic intent classifier in the policy engine. Tool misuse (ASI02) is addressed by capability sandboxing and the MCP gateway. Identity abuse (ASI03) is handled through DID-based identity with behavioural trust scoring. Supply chain risks for agent plugins are addressed through Ed25519 plugin signing and manifest verification.

For security teams building or governing autonomous AI agents that integrate with enterprise systems via MCP servers, this toolkit provides the governance infrastructure that no other open source project currently addresses at this level of completeness. The EU AI Act’s high-risk AI obligations, which take effect in August 2026, explicitly require audit trails and human oversight mechanisms for autonomous systems: the Agent Governance Toolkit generates the policy enforcement logs needed to satisfy these requirements. This makes it an essential component of a comprehensive AI security programme for any organisation deploying agentic workloads in regulated environments.

Primary use: Runtime governance for autonomous AI agents, agentic AI security Licence: MIT Key OWASP coverage: All 10 OWASP ASI Agentic Top 10 risks Agentic support: Purpose-built for agentic AI EU AI Act alignment: Yes (policy logs and human oversight mechanisms)

Microsoft Presidio: PII Detection for AI Pipelines

Microsoft Presidio is an open-source PII (personally identifiable information) detection and anonymisation library maintained by Microsoft under the MIT licence. Among the open-source cybersecurity tools specifically designed for AI pipeline security, Presidio addresses a specific but high-consequence risk: sensitive data being injected into model context through retrieval-augmented generation (RAG) pipelines and subsequently appearing in model outputs or being accessible to adversarial prompt injection attacks.

Presidio provides 20 or more entity recognisers covering names, email addresses, phone numbers, credit card numbers, national identifiers, medical record numbers, and custom entity types. It operates as a middleware layer: incoming documents retrieved for RAG pipelines can be scanned and anonymised before injection into model context, and model outputs can be scanned before delivery to users. Security professionals integrating Presidio into AI pipelines can configure entity recognition thresholds and anonymisation strategies to balance data utility with protection requirements.

The library is lightweight, production-tested at Microsoft scale, and extensible via custom recogniser plugins. For Australian organisations subject to the Privacy Act 1988 and the Notifiable Data Breach (NDB) scheme, Presidio’s structured anonymisation logs provide evidence of proactive data protection measures applied to AI workflows. This makes it one of the most important open-source tools in any secure AI deployment stack for regulated industries.

Primary use: PII detection and anonymisation in AI pipelines, RAG security Licence: MIT Key OWASP coverage:LLM02 (Sensitive Information Disclosure), LLM06 CI/CD integration: Yes, via Python library

LLM Guard: Input/Output Scanning for Production AI

LLM Guard is an Apache 2.0 open source security toolkit originally developed by Protect AI, providing lightweight input and output scanning for LLM-powered applications. It focuses on four protection areas: PII detection, prompt injection resistance, harmful content filtering, and secrets/credential detection in model outputs. It is one of the most practically deployed cybersecurity tools in the open source AI security category.

The toolkit is designed for low-latency production deployment. Scanning operations run in milliseconds rather than seconds, making it suitable for real-time API-level interception without introducing perceptible latency to end users. LLM Guard functions as a complementary layer to NeMo Guardrails: where NeMo provides programmable dialogue and topic control, LLM Guard provides rapid binary classification for security-relevant signals in individual inputs and outputs.

For AI security platform integrations, LLM Guard’s structured output format makes it straightforward to integrate into SIEM pipelines, enabling security operations teams to monitor AI application security signals alongside other telemetry. This supports security operations workflows where AI-generated alerts need to be correlated with other security events across the environment.

Primary use: Runtime LLM input/output scanning, secrets and PII filtering Licence: Apache 2.0 Key OWASP coverage: LLM01, LLM02, LLM06 Agentic support: Partial (input scanning for agent tool inputs) Integration:SIEM-compatible structured output

Comparison Table: Open Source AI Security Tools at a Glance

Tool	Licence	Phase	Primary Threat	Agentic AI	CI/CD	OWASP Coverage
Garak	Apache 2.0	Pre-deploy	LLM vulnerabilities	Yes (v0.15.0)	Yes	LLM01, LLM02, LLM03
Promptfoo	MIT	Pre-deploy	Prompt injection, jailbreaks	Yes	Native	LLM01, LLM06, LLM08
DeepTeam	Apache 2.0	Pre-deploy + Runtime	Agentic attack simulation	Purpose-built	Yes	OWASP ASI, LLM01
ART	MIT	Pre-deploy	Adversarial ML (all types)	Partial	Python	MITRE ATLAS aligned
NeMo Guardrails	Apache 2.0	Runtime	Behavioural guardrails	Yes	Framework	LLM01, LLM08
Agent Governance Toolkit	MIT	Runtime	Agentic policy enforcement	Purpose-built	No	All 10 OWASP ASI
Presidio	MIT	Runtime	PII in AI pipelines	Partial	Yes	LLM02, LLM06
LLM Guard	Apache 2.0	Runtime	Input/output scanning	Partial	Yes	LLM01, LLM02

A Practitioner’s Recommended Stack by Maturity

Not every organisation needs all eight tools on day one. The following phased approach aligns tool adoption with security maturity and enables developers and security professionals to build incrementally rather than attempting a full deployment before the security team has operational familiarity with each layer.

Starter stack (weeks 1 to 4): Garak and NeMo Guardrails. Garak establishes a baseline vulnerability assessment of any LLM or AI application before deployment. NeMo Guardrails provides the runtime protection layer for production. Together, these two tools address the most critical pre-deployment and runtime risks at minimal operational overhead. This combination covers the primary security context for most LLM application deployments.

Intermediate stack (month 1 to 3): Add Promptfoo for CI/CD integration and Microsoft Presidio for RAG pipeline data protection. At this stage, AI security testing becomes part of the standard release workflow rather than a periodic assessment activity. Presidio addresses the data protection requirements that most regulated Australian organisations face when personal information flows through AI pipelines. Developers and security teams gain shared visibility into security signals across the delivery lifecycle.

Advanced / enterprise stack (month 3 onward): Add DeepTeam for agentic AI red teaming, the Microsoft Agent Governance Toolkit for autonomous agent governance, ART for adversarial ML testing of non-LLM models, and LLM Guard as a lightweight runtime scanning layer. At full maturity, this stack provides comprehensive security coverage across the OWASP LLM Top 10 and the OWASP Agentic Top 10. This is the stack appropriate for organisations deploying autonomous AI agents in regulated or high-risk operational environments.

Supply Chain Trust: What the Trivy Incident Changes

In March 2026, the open source vulnerability scanner Trivy was compromised by the threat actor group TeamPCP in a sophisticated supply chain attack. The attackers gained access to Aqua Security’s infrastructure, retroactively poisoned 76 of 77 release tags by repointing them to malicious code, and distributed a credential-stealing payload through official channels including Docker Hub, the trivy-action GitHub Action, and the setup-trivy GitHub Action. The malicious code ran silently before the real scanner, so CI/CD workflows appeared to complete normally while credentials were exfiltrated.

This incident is directly relevant to every security team consuming open source security tooling. The trust model that allows security tools to be consumed from public registries with pinned version tags is the same trust model that the attack exploited. Several lessons apply specifically to the open source AI security tools in this guide:

• Pin to commit SHAs, not version tags. Git tags are mutable and can be repointed, as the Trivy compromise demonstrated. GitHub Actions workflows should reference actions by their full commit SHA rather than a tag like v3 or latest.
• Verify signatures and checksums independently. For tools installed via pip or npm, verify package checksums against independently published hashes before using them in security-sensitive workflows.
• Treat your AI security tooling as part of your supply chain security programme. An AI bill of materials (AI-BOM) should include not only the models your systems use but also the security tools operating in your CI/CD pipelines.
• Monitor for anomalous behaviour in CI/CD jobs. The Trivy attackers ran credential exfiltration before the legitimate scanner executed. Baseline monitoring of what CI/CD security jobs actually do at the network and file system level provides early detection of compromised tooling.

The tools in this guide are all actively maintained with strong community oversight, but the Trivy incident demonstrates that no open source project is immune to supply chain compromise. Operational hygiene for consuming open source security tooling is as important as the tooling itself.

Mapping to OWASP Frameworks: LLM Top 10 and the Agentic Top 10

OWASP LLM Top 10 (2025 edition) remains the primary reference for securing large language model applications. The 2025 list retains LLM01 Prompt Injection as the highest-severity risk, followed by LLM02 Sensitive Information Disclosure, LLM03 Supply Chain Vulnerabilities, LLM04 Data and Model Poisoning, LLM05 Improper Output Handling, LLM06 Excessive Agency, LLM07 System Prompt Leakage, LLM08 Vector and Embedding Weaknesses, LLM09 Misinformation, and LLM10 Unbounded Consumption.

OWASP Top 10 for Agentic Applications (ASI, December 2025) addresses the distinct threat model that emerges when LLMs can plan, persist, and delegate across tools and systems. The Agentic Top 10 was developed through collaboration with more than 100 industry experts and addresses risks that the LLM Top 10 does not fully capture: goal hijacking (ASI01), unsafe agent actions (ASI02), agent impersonation (ASI03), memory poisoning (ASI04), tool misuse (ASI05), cascading hallucinations (ASI06), intent breaking (ASI07), data exfiltration (ASI08), privilege escalation (ASI09), and over-reliance on agent decision-making (ASI10).

The following table maps each open source tool to the primary OWASP risk categories it addresses:

OWASP Risk	Primary Tool	Supporting Tool
LLM01 Prompt Injection	Garak, Promptfoo	LLM Guard
LLM02 Sensitive Info Disclosure	Presidio	Garak, LLM Guard
LLM03 Supply Chain	ART (model integrity)	Agent Governance Toolkit
LLM06 Excessive Agency	DeepTeam	NeMo Guardrails
ASI01 Goal Hijacking	Agent Governance Toolkit	DeepTeam
ASI02 Unsafe Agent Actions	Agent Governance Toolkit	NeMo Guardrails
ASI04 Memory Poisoning	DeepTeam	Agent Governance Toolkit
ASI05 Tool Misuse	Agent Governance Toolkit	Promptfoo