LLM red teaming tools gain traction

The LLM security testing toolchain is quietly consolidating around a handful of open-source frameworks, and teams that don't adopt them before hitting production are flying blind.

• –Garak (NVIDIA) is the most comprehensive model-level scanner — 120+ probe modules targeting jailbreaks, prompt injection, data leakage, and toxicity — but it tests models in isolation, not full app stacks
• –promptfoo bridges the gap: it tests complete LLM systems including RAG pipelines and agent architectures, dynamically generating thousands of attack variants tailored to your app context, with CI/CD integration
• –DeepTeam (confident-ai) is the newest entrant — open-source, Python-native, supports OWASP Top 10 for LLMs and NIST AI RMF out of the box, and doesn't require a pre-built dataset since attacks are simulated dynamically
• –The "manual testing works at the start" pattern is universal — but the window before real users hit edge cases is shrinking as LLM features ship faster
• –No tool currently dominates: teams are mixing evals (LangSmith) with security probing (Garak/promptfoo) depending on whether they're testing model behavior vs. application-level vulnerabilities