Week 11 · Phase 5: Practice & career

AI red teaming & tools

Test AI systems like an attacker, with permission, and report what you find.

Time this week: 8–10 hours

Goal for this week
By Sunday you can explain how AI red teaming works, run a simulated garak-style scan, read the results, and write a short findings report.
Why it matters for AI security
Red teaming is the hands-on path. Companies need people who can test their chatbots and agents before attackers do. Tools like garak (NVIDIA) and PyRIT (Microsoft) automate a lot of it.

Start here: this week's labs

Red Team Scanner Lab (garak-style, simulated)
Type real-looking scanner commands in a fake terminal, run probes against two simulated models, read the results and build a findings report.
Open lab

How a scanner like garak works

Probesattack prompts
Target modelgets the prompts
Responseswhat it said
Detectorsdid it fail?
Reportpass / fail rates

Key ideas

AI red teaming

Testing an AI system by attacking it on purpose, with permission, to find weaknesses: injection, leaks, harmful output, unsafe tool use.

Rules of engagement

Written permission and scope: what you can test, when, and what's off limits. Never test systems you don't have permission for.

garak

Free open-source LLM vulnerability scanner from NVIDIA. It sends many 'probes' (attack prompts) and 'detectors' judge if the model failed.

PyRIT

Python Risk Identification Tool from Microsoft (Azure). A framework for automating red-team attacks, including multi-turn conversations.

Attack success rate

How many attack attempts worked, out of how many tried. A simple way to compare models and defenses.

Findings report

What you found, how serious it is, how to reproduce it, and how to fix it. The report is the real product of red teaming.

Day-by-day plan

Tick each day when you finish it. Your ticks are saved in this browser.

Mini game: Red team or not allowed?

Practice outside this course (free)

garak (NVIDIA)
Free LLM vulnerability scanner. pip install garak.
PyRIT (Microsoft)
Free red-teaming framework for generative AI.
Gandalf by Lakera
Keep practicing prompt attacks.
PortSwigger Web LLM attacks
Try 'Indirect prompt injection' and the AI agent labs.

These links need the internet. Everything else on this page works offline.

Exercises & worksheet

Exercise deck (PPTX) Worksheet (DOCX)

Weekly quiz

Finished the week?
Mark it done to fill your progress bar on the course home.
← Week 10: Enterprise AI controls: SSO, DLP, CASB/SSE Week 12: Capstone, certifications & career →