What Is Pose Estimation?

Pose estimation, often called human pose estimation, is a computer vision task that locates the key points of a body in an image or video frame, such as the nose, eyes, shoulders, elbows, wrists, hips, knees and ankles. Joining those keypoints gives a simple skeleton for each person in view. The widely used COCO dataset defines 17 body keypoints, and some models add hands, feet and face points for finer detail.

Simple person detection only says someone is there. Pose estimation shows where their arms and body are, so a safety system can tell whether a hand is reaching towards a robot or an operator is bending awkwardly. Followed from frame to frame, the skeleton turns into movement: a step, a lift, a fall or a gesture. The same idea works for animals, vehicles and robots, though people are by far the most common subject.

A deep learning model, trained on thousands of images where people’s joints have been labeled by hand, predicts where each keypoint is and how confident it is. Models follow one of two strategies when several people are in view.

Well-known models include OpenPose, an early bottom-up model, HRNet and ViTPose for accuracy, and MediaPipe BlazePose, YOLO pose models and RTMPose for real-time use.

2D pose estimation places keypoints on the flat image, in pixels. It is fast and needs only one ordinary camera, and it is enough for many tasks, such as counting lifts or spotting a raised arm. 3D pose estimation adds depth, giving each joint a position in space. That is what a safety system needs to know how far a hand is from a moving robot. Depth can come from a stereo camera, from several synchronized cameras viewing the same area, or from a model that estimates depth from a single image, with lower accuracy.

Pose estimation suits any job where how a person moves matters more than who they are. Our explainer on video analytics covers the wider family of techniques it belongs to.

For safety and control, pose estimation has to run beside the cameras. A robot cell cannot wait for video to travel to the cloud and back, and many sites do not want images of staff leaving the building. Running the model on an edge computer keeps the response in milliseconds and lets the system store only skeletons, not faces.

The compute load depends on the model, the resolution, the number of cameras and how many people are in view. A single camera with a real-time model runs comfortably on Jetson Orin Nano computers, while several high-resolution cameras with 3D fusion need far more. Our GMSL cameras connect directly to Jetson systems with low latency and synchronized capture. The human-robot collaboration case study runs pose estimation on Jetson with GMSL cameras.

Optimizing the model matters as much as the hardware. Converting it to INT8 with TensorRT and choosing an input resolution that just covers the working area often doubles the frame rate, leaving room for detection, tracking and the safety logic on the same computer.

Pose models struggle when the body is hidden. Occlusion by machinery or other people, loose clothing, high-visibility jackets, unusual camera angles and poor light all reduce accuracy, and models trained on public datasets of everyday scenes may do worse on a factory floor. Accuracy is usually measured with object keypoint similarity, which scores how close predicted joints are to the labeled ones. Fine-tuning on images from the real site, and placing cameras to see the working area clearly, usually help more than switching to a larger model.

We build edge AI computers for pose estimation with the cameras, AI software and optimized models integrated as standard, and test the full vision chain before the system ships.