A moderation confidence score is a number a moderation system assigns to its own decision, showing how certain it is that a piece of content violates policy. A high score means the system is confident in its call, and a low score means the decision should go to a person.

‍

‍Why it matters

Automated systems don't just say yes or no, they usually calculate how sure they are, and that number decides what happens next. A system might be very sure a post is spam and remove it automatically, but unsure about something ambiguous, where a human reviewer should decide. Without confidence scores, a platform has to treat every automated decision the same way, which either wastes reviewer time on obvious cases or risks real mistakes on uncertain ones.

‍

‍How thresholds get set

A threshold is set for each policy category. Content scoring above it is actioned automatically, content below it goes to a human queue, and the cut-off is chosen by testing against reviewed samples. Thresholds get revisited regularly, based on how often human reviewers agree with the system's high-confidence decisions.

‍

‍A score is not a probability

A score of 0.9 doesn't automatically mean the system is right 90% of the time. That's only true if the model is well calibrated, meaning its confidence matches how often it's actually correct. Many models are overconfident by default, so teams check the scores against human-reviewed samples before trusting them to drive decisions.