Instance segmentation is a computer vision technique that identifies and outlines each individual object in an image separately, even when multiple objects belong to the same category. Unlike semantic segmentation, which labels all pixels of a category as one group, instance segmentation tells the difference between, say, the third car in a photo and the fourth.

‍

‍Why it matters

‍ Knowing that a group of pixels is people isn't enough for a lot of real applications, a retail store counting customers, a self-driving car tracking multiple pedestrians, or a sports analytics tool following individual players all need to know where one specific object ends and the next one of the same type begins. Semantic segmentation alone would just show one big blob labeled person if several people overlap in a crowd, which loses exactly the information these applications need.

‍

‍How it's labeled

Building this kind of dataset means tracing a separate outline for every individual object of interest, even when two objects of the same category overlap or touch, which is meaningfully harder than semantic segmentation's single shared mask per category. Tools like CVAT and Labelbox support instance-level masks directly, and annotators typically need clearer guidelines on how to handle partial occlusion, deciding where one overlapping object ends and another begins, since that's where most of the real disagreement happens.

‍

‍Where it's used

Retail analytics uses this to count individual customers or track products separately even when items overlap on a shelf. Sports analytics relies on it to follow individual players through a game without losing track of who's who when players cluster together. Autonomous vehicle systems use it too, distinguishing one pedestrian from another in a crowd rather than treating a group as a single object.