Data bias is a systematic skew in a dataset that causes a model trained on it to perform unevenly or unfairly across different groups, situations, or types of input, rather than an occasional random error. Unlike random noise, bias points in a consistent direction, which means it shows up reliably in a model's behavior rather than washing out over time.

‍

‍Why it matters

Bias usually enters a dataset quietly, through who happened to be available to collect data from, what data was easiest to gather, or which examples annotators were more familiar with, rather than through any deliberate decision. A facial recognition system trained mostly on one demographic will predictably perform worse on others, not because anyone intended that outcome, but because the training data simply didn't represent everyone equally. This matters enormously in high stakes applications, hiring, lending, healthcare, content moderation, where a biased model can cause real, unequal harm at scale rather than being a purely academic concern.

‍

‍How teams check for it

‍Teams typically check for bias by measuring model performance separately across different subgroups rather than relying on one overall accuracy number, since a strong average can hide a much weaker result for a specific group. Data stratification plays a direct role here too, deliberately ensuring underrepresented groups have proportional representation in a dataset rather than being an afterthought, since the fix usually has to happen at the data level, not just by adjusting the model after the fact.