Adversarial moderation is moderation built to deal with people who deliberately try to get around it. It assumes some users will change their behavior to avoid detection once they learn what gets caught.

‍

‍Why it matters

Moderation is a cat-and-mouse game. A static keyword list works for a while, then users swap letters for symbols, add spaces, misspell on purpose, put text inside images, or use coded words. Algospeak is the term for the coded language people develop to avoid moderation, like replacing a blocked word with a harmless-sounding stand-in. Any filter that isn't updated will decay as these workarounds spread.

‍

‍How teams respond

Common countermeasures include normalizing text before matching, so swapped characters and stray spaces are collapsed, and reading text inside images. Reviewers feed newly discovered evasion terms back into the lists, and teams run adversarial testing, where someone tries to break the system on purpose, before real users do. It overlaps with red teaming for AI models, though here the target is a moderation system.