AI red teaming & tools
Test AI systems like an attacker, with permission, and report what you find.
Time this week: 8–10 hours
By Sunday you can explain how AI red teaming works, run a simulated garak-style scan, read the results, and write a short findings report.
Red teaming is the hands-on path. Companies need people who can test their chatbots and agents before attackers do. Tools like garak (NVIDIA) and PyRIT (Microsoft) automate a lot of it.
Start here: this week's labs
How a scanner like garak works
Key ideas
AI red teaming
Testing an AI system by attacking it on purpose, with permission, to find weaknesses: injection, leaks, harmful output, unsafe tool use.
Rules of engagement
Written permission and scope: what you can test, when, and what's off limits. Never test systems you don't have permission for.
garak
Free open-source LLM vulnerability scanner from NVIDIA. It sends many 'probes' (attack prompts) and 'detectors' judge if the model failed.
PyRIT
Python Risk Identification Tool from Microsoft (Azure). A framework for automating red-team attacks, including multi-turn conversations.
Attack success rate
How many attack attempts worked, out of how many tried. A simple way to compare models and defenses.
Findings report
What you found, how serious it is, how to reproduce it, and how to fix it. The report is the real product of red teaming.
Day-by-day plan
Tick each day when you finish it. Your ticks are saved in this browser.
Mini game: Red team or not allowed?
Practice outside this course (free)
Free LLM vulnerability scanner. pip install garak.PyRIT (Microsoft)
Free red-teaming framework for generative AI.Gandalf by Lakera
Keep practicing prompt attacks.PortSwigger Web LLM attacks
Try 'Indirect prompt injection' and the AI agent labs.
These links need the internet. Everything else on this page works offline.
Exercises & worksheet
Weekly quiz
Mark it done to fill your progress bar on the course home.