Krippendorff's Alpha measures agreement among multiple raters while correcting for chance, like Cohen's Kappa, but it works with more than two raters, missing ratings, and different kinds of data, such as categories, rankings, or numeric scores, in one formula.

‍

Why it matters

Cohen's Kappa stops being usable once a project has three or more raters, and Fleiss' Kappa needs every item to get the same number of ratings. Real projects often don't work that way, since some raters skip items or review different amounts. Krippendorff's Alpha was built for those messier situations, which is why it's often the preferred choice for larger or more complex annotation projects.

‍

What the number means

Interpretation follows a similar scale to Kappa. Krippendorff's own guidance treats values of 0.80 and above as reliable and values from about 0.67 to 0.80 as enough only for tentative conclusions, though the right threshold depends on the stakes. The calculation is more involved than percent agreement, so most teams use an existing statistics library instead of working it out by hand.