The headline that security bots can be tricked into launching attacks did not appear out of nowhere. It came out of a year of research showing that agentic AI systems, given the wrong document, can be steered into decisions the operator never sanctioned. If your team ships anything that lets a language model act, one of these AI red teaming apps needs to be in the toolchain.
We tested seven apps that run on desktop workstations for engineers who want to stress-test their own models and agents before an outside adversary does. The list mixes command-line frameworks, GUI-based scanners, and libraries meant to run inside CI.
What to look for in an AI red teaming app
The category is young and the tools are not interchangeable. Look for:
- A concrete attack catalogue. Prompt injection, data exfiltration, prompt leaking, jailbreaks, and multi-turn manipulation should all appear.
- A way to plug into your model. Local models via OpenAI-compatible endpoints, hosted APIs, and custom Python callables all matter.
- Reproducible reports. A written finding a security team can act on.
- Automation. The tool needs to run in CI or on a nightly cron, not just a laptop console.
- Live guardrails or just testing. Some tools scan; others block at inference time. Know which one you need.
Quick comparison
| App | Best for | Platforms | Free plan | Standout |
|---|---|---|---|---|
| Promptfoo | Continuous eval and red teaming | Windows, macOS, Linux | Fully free, open-source | CI-friendly config, growing attack pack |
| Garak | Broad vulnerability scanning | Windows, macOS, Linux | Free, open-source | NVIDIA-backed, catalog of probes |
| PyRIT | Microsoft’s automated framework | Windows, macOS, Linux | Free, open-source | Multi-turn attack orchestration |
| NeMo Guardrails | Live guardrail policy | Windows, Linux | Free, open-source | Colang policy language for real-time rules |
| Lakera Guard | Hosted guardrails and scanning | Cloud, plus SDKs | Free tier | Managed prompt-injection detection |
| Adversarial Robustness Toolbox | Broader ML attacks and defences | Windows, macOS, Linux | Free, open-source | Not just LLMs, also vision and tabular |
| Rebuff | Injection detection tuned for LLMs | Windows, macOS, Linux | Free tier | Vector-store canary detection |
The apps
1. Promptfoo — Best for continuous eval
Promptfoo turns AI red teaming into a CI job. The config is a single YAML file that describes prompts, providers, and attacks, and the CLI runs the same suite against local models via Ollama or hosted APIs. The red-teaming pack ships jailbreaks, data-exfiltration probes, and prompt-injection tests you can extend.
Where it falls short: The reporting UI in the free tier is functional; team dashboards are a paid add-on.
Pricing:
- Free: Full CLI and open-source library
- Paid: Managed team tier for shared runs and org dashboards
Platforms: Windows, macOS, Linux (Node.js runtime)
Download: promptfoo.dev
Bottom line: The pick for a team that wants red teaming to run every night alongside unit tests.
2. Garak — Best broad vulnerability scanner
Garak is NVIDIA’s LLM vulnerability scanner. It ships a catalogue of probes (prompt injection, data leakage, malware generation, hallucination-under-pressure) and reports findings in a format that maps to a familiar security workflow.
Where it falls short: The output is text-heavy; presenting it to a business owner needs a second-pass summary.
Pricing:
- Free: Full open-source scanner
- Paid: None
Platforms: Windows, macOS, Linux (Python)
Download: GitHub
Bottom line: The pick for security engineers running a first pass against a new model to see what stands out.
3. PyRIT — Best automated attack orchestration
PyRIT is Microsoft’s Python Risk Identification Toolkit. It automates multi-turn attacks, chaining prompts across a conversation to escalate the way a real adversary would, and it plugs into any model behind a callable.
Where it falls short: Steep initial config. Best used by teams with a security engineer who reads Python fluently.
Pricing:
- Free: Full open-source library
- Paid: None
Platforms: Windows, macOS, Linux
Download: GitHub
Bottom line: The pick for an internal red team writing scripted, escalating attacks rather than one-shot probes.
4. NeMo Guardrails — Best live guardrail
NeMo Guardrails is NVIDIA’s runtime policy engine. Colang, its policy language, describes what a model is and is not allowed to say or do, and the framework enforces those rules between the prompt and the reply in real time.
Where it falls short: Not a scanner, a runtime. It is complementary to Promptfoo or Garak rather than a substitute.
Pricing:
- Free: Full open-source library
- Paid: None
Platforms: Windows, Linux (Python)
Download: GitHub
Bottom line: The pick to actually block a bad reply in production, once red teaming has surfaced the vector.
5. Lakera Guard — Best hosted guardrail
Lakera Guard ships as a hosted service with SDKs for major languages. Prompt-injection detection is the flagship, and the team publishes a research feed of newly discovered prompt-injection patterns that flows into the detector.
Where it falls short: Managed only. Anyone with strict data residency requirements needs to move up a tier to a self-hosted deployment.
Pricing:
- Free: Free tier with monthly request cap
- Paid: Team and enterprise tiers, self-hosted option at the top
Platforms: Cloud, SDKs for all major runtimes
Download: lakera.ai
Bottom line: The pick for teams that would rather buy prompt-injection detection than build it.
6. Adversarial Robustness Toolbox — Best beyond LLMs
Adversarial Robustness Toolbox, maintained by the Linux Foundation and originally released by IBM Research, ships attacks and defences for the wider machine-learning surface: vision models, tabular classifiers, and speech systems. LLM support is present but not the pitch.
Where it falls short: Broader than most LLM-focused teams need. Learning curve is steepest of any on this list.
Pricing:
- Free: Full open-source library
- Paid: None
Platforms: Windows, macOS, Linux (Python)
Download: GitHub
Bottom line: The pick for a team securing multiple ML modalities, not just an LLM.
7. Rebuff — Best injection-focused detection
Rebuff is a focused prompt-injection detection library with heuristic checks, LLM-based checks, and a vector-store “canary” that logs when a prompt is leaked into a data store it was not supposed to touch.
Where it falls short: Narrow focus. Great at prompt injection, silent on other attack classes.
Pricing:
- Free: Open-source library
- Paid: Hosted managed tier with dashboards
Platforms: Windows, macOS, Linux
Download: GitHub
Bottom line: The pick for a project whose main threat surface is a chatbot with user-supplied documents in the context.
How to pick the right one
If you want continuous red teaming in CI: Promptfoo. YAML in, findings out, runs on the same worker as your unit tests.
If you want a broad vulnerability scan: Garak. It is the one that mirrors a security-engineer workflow most closely.
If you want scripted multi-turn attacks: PyRIT. Microsoft’s toolkit assumes escalation and treats it as a first-class concern.
If you want to block in production: NeMo Guardrails or Lakera Guard. Pick self-hosted or managed based on data-residency needs.
If your surface includes vision and tabular models: Adversarial Robustness Toolbox. Broader coverage, higher setup cost.
If prompt injection is the specific concern: Rebuff. Focused, targeted, easy to slot into an existing stack.
FAQ
What is AI red teaming? The practice of adversarial testing against an AI system to surface behaviours the operator did not want, before an outside attacker does. It borrows the term from network security red-team engagements.
Do I need AI red teaming if my model is a hosted API? Yes. Prompt injection, data exfiltration through tools, and hallucinations that generate liability all cross the API boundary. Hosted models still need application-level testing.
Are these tools free? Most on this list are free and open-source. Lakera Guard and the managed tiers of Promptfoo and Rebuff have paid options.
Which app is easiest to start with? Promptfoo. A YAML file, a provider, and one command produces a first result inside an hour.
Can these tools test my own local model? Yes. Every open-source tool on this list supports OpenAI-compatible endpoints, so a local Ollama or LM Studio server is a valid target.
Do these apps generate a report I can share with a security team? Promptfoo, Garak, PyRIT, and Rebuff all produce reports intended for triage. The formats vary. Managed tiers add dashboards and shared runs.
