Keypoint annotation is a labeling method that marks specific individual points on an object, like the joints of a human body, the corners of an eye, or the tip of a machine part, rather than outlining the object's full shape. Each point typically gets connected in a defined skeleton structure that captures pose or precise structural detail.‍

Why it matters

A bounding box or segmentation mask can tell a model where a person is in an image, but it can't tell the model how that person's body is actually positioned, are their arms raised, are they crouching, which way is their head turned. Keypoint annotation exists specifically to capture that structural detail, which matters enormously for applications like pose estimation, sports analytics, or fitness apps that need to understand body movement rather than just detect that a person is present.

‍How it's labeled

Annotators place individual points at predefined body locations, elbows, wrists, knees, following a fixed skeleton template so every labeled image uses the same point structure and can be compared consistently. Tools like CVAT support keypoint annotation directly, and the trickiest part is usually labeling points that are partially hidden or occluded, where annotators have to estimate a joint's likely position rather than mark something they can actually see.

‍Where it's used

Sports analytics uses this constantly, tracking joint positions to analyze an athlete's form or technique frame by frame. Fitness and AR apps rely on it to recognize whether someone is performing an exercise correctly in real time. Robotics uses the same idea in reverse, giving a robot arm keypoints to track so it understands the exact position of what it's trying to grip.