Quick links
Skip to main menu
Skip to content

2026.11.06 Tech

Multi-HMR 2: AI That Understands the Positions and Movements of Multiple People in 3D

NAVER LABS Europe’s Multi-HMR 2 is an AI model that reconstructs the poses, body shapes, and positions of multiple people in an image in 3D, and tracks the movements of the same person in video. The underlying technology is Human Mesh Recovery (HMR). It reproduces a person’s body in a photo or video as a three-dimensional surface structure, representing not only the positions of the joints but also the shape of the body.

​

Previously, it was common to first detect people and then recover them one by one. In 2024, NAVER LABS Europe introduced an approach that recovers multiple people at once through Multi-HMR. Multi-HMR 2 builds on this by estimating spatial positions more accurately and adding the ability to follow the same person across a video.

​

Multi-HMR 2 has five key features.

​

  • Simultaneous 3D reconstruction of multiple people

  • Distance and position estimation without camera information

  • Improved recognition of overlapping or occluded people

  • Representation of diverse ages and body types

  • Tracking the movements of the same person

​

​

More Accurately Determining Where People Are

Multi-HMR 2 directly estimates camera information from the image and determines how far each person is from the camera in actual metric distances. Even without knowing the camera information in advance, it can more accurately recover people’s 3D positions and the distance relationships between them. In addition, because it analyzes the entire image at once, it reliably distinguishes and reconstructs people even in scenes where several people overlap or parts of their bodies are occluded

Multi-HMR 2 processing pipeline

Multi-HMR 2 applies Anny, a parametric 3D body model developed in-house, to accurately reconstruct people of various ages and body types. Anny can represent everyone from infants to the elderly, reducing the limitation of adult-centered body models, which reconstructed children as small adults. By reflecting the different body proportions of each person, it recovers body shapes naturally and estimates 3D positions more accurately.

​

​

Tracking the Same Person and Helping Robots Perceive Their Surroundings

Another key new feature of Multi-HMR 2 is tracking the same person. It goes beyond reconstructing people in still images in 3D: across consecutive frames of a video, it follows where the same person moves.

This allows it to continuously track each person’s position and movement even when several people cross paths or are briefly occluded. Such information can help robots move in the same space as people. This is because a robot can use it to plan its path while taking into account the positions and movements of people nearby, or to keep recognizing the person it is providing a service to.

NAVER LABS Europe plans to continue research on reconstructing hands and faces in greater detail, and to further improve overall accuracy and stability by using temporal information and images from multiple viewpoints.

​

Learn more

​

Paper >

Code >

Demo >

​

Related Articles