Sampling methodology is the strategy a team uses to select a smaller subset of data, decisions, or interactions to review, test, or check, instead of examining everything in a much larger pool. The specific method chosen determines whether that smaller subset actually represents the whole accurately or ends up skewed.

‍

‍Why it matters

Reviewing everything isn't practical once volume gets large, so teams need a way to check a portion and still trust the results apply to the whole. A poorly chosen sample can hide real problems, if a team only samples the easiest, most common cases, they'll miss issues that only show up in rare or unusual ones.

‍

‍How it's used in QA

In practice, sampling methodology usually decides how QA pulls its review batches, whether that's a straight random pull or a stratified pull that makes sure rare categories get checked just as often as common ones. Getting this right is what makes an accuracy score from a sample actually mean something for the whole, rather than just reflecting the easy cases.

‍

‍Common approaches

Random sampling picks items completely at random, which works fine when the pool is fairly uniform. Stratified sampling breaks the data into groups first, by category or difficulty, then samples proportionally from each group. Systematic sampling picks every Nth item, simpler to set up, but it can accidentally line up with a hidden pattern if a team isn't careful.