Week 6 · Phase 3: Attacks on AI

Prompt injection & jailbreaks

The number one AI attack. You'll attack a bot, then defend it.

Time this week: 8–10 hours

Goal for this week
By Sunday you can explain direct vs indirect prompt injection and jailbreaks, beat 5 levels of the simulated bot, and pick defenses that actually help.
Why it matters for AI security
Prompt injection is LLM01, the top risk in the OWASP Top 10 for LLM Applications 2025. Anyone securing ChatGPT, Claude or Gemini at work has to understand it deeply.

Start here: this week's labs

Prompt Injection Lab: steal the secret
A simulated bot with 5 levels of defenses guards a password. Break it with real techniques, then switch to Defender mode.
Open lab

Defense in depth: layers around the model

Monitoring, logging & incident response
Least privilege + human approval for actions
Input filter + output filter
System prompt (helps, but is NOT a security wall)
The LLM

Key ideas

Direct prompt injection

The user types instructions that override the app's rules: 'Ignore previous instructions and...'.

Indirect prompt injection

The instructions hide inside content the AI reads: a web page, email, PDF, or calendar invite.

Jailbreak

A prompt that tricks the model into ignoring its safety training, often with role-play or 'hypothetical' stories.

System prompt leakage

Getting the bot to reveal its hidden instructions. LLM07 in the OWASP list. Never put secrets in a system prompt.

Why it's hard to fix

The model sees instructions and data as the same kind of text. There is no perfect fix yet, only layers of defense.

Defense in depth

Input filters + output checks + least privilege + human approval + monitoring. No single layer is enough.

Day-by-day plan

Tick each day when you finish it. Your ticks are saved in this browser.

Mini game: Direct, indirect or jailbreak?

Practice outside this course (free)

Gandalf by Lakera
Free game: trick Gandalf into revealing a password. 7+ levels.
OWASP LLM01: Prompt Injection
The official description, examples and defenses.
HackAPrompt playground (Learn Prompting)
Prompt hacking challenges and competitions. Check what is open now.

These links need the internet. Everything else on this page works offline.

Exercises & worksheet

Exercise deck (PPTX) Worksheet (DOCX)

Weekly quiz

Finished the week?
Mark it done to fill your progress bar on the course home.
← Week 5: RAG, agents & tools (+ a little Python) Week 7: OWASP Top 10 for LLMs & agent risks →