AI IntelligenceAug 22, 2026AI Intelligence
Article
If you're using Prompt Guard 2 to catch injections, check what it actually catches on your own attacks first (mine: 22.8%): A user...
Tested Meta's Prompt Guard 2 (86M, open-weight) on out-of-distribution prompt-injection attacks; at the default operating point it caught only 22.8% of fresh HackAPrompt injections. The takeaway: benchmark injection classifiers on your own attacks instead of assuming training-set performance.
Frontier EditorialSource: Reddit r/PromptEngineering
01
Source Brief
If you're using Prompt Guard 2 to catch injections, check what it actually catches on your own attacks first (mine: 22.8%): A user tested Meta's Prompt Guard 2 (86M, open-weight) on out-of-distribution prompt-injection attacks; at the default operating point it caught only 22.8% of fresh HackAPrompt injections. The takeaway: benchmark injection classifiers on your own attacks instead of assuming training-set performance.