[CVPR 2024] BodyMAP: Jointly Predicting Body Mesh and 3D Applied Pressure Map for People in Bed

Abstract: Accurately predicting the 3D human posture and the pressure exerted on the body while resting in bed, visualized as a body mesh (3D pose & shape) with a 3D pressure map, holds significant promise for healthcare applications, particularly, in the prevention of pressure ulcers. Current methods focus on singular facets of the problem---predicting only 2D/3D poses, generating 2D pressure images, predicting pressure only for certain body regions instead of the full body, or forming indirect approximations to the 3D pressure map. In contrast, we introduce BodyMAP, that jointly predicts the human body mesh and 3D applied pressure map across the entire human body using a depth image and its associated 2D pressure image from a pressure-sensing mattress. The 3D pressure map is represented as a pressure value at each mesh vertex, enabling precise localization of high-pressure regions on the body. Additionally, we present BodyMAP-WS, a new formulation of pressure prediction in which we implicitly learn pressure in 3D by aligning sensed 2D pressure images with a differentiable 2D projection from predicted 3D pressure maps. In evaluations with real-world human data, our method outperforms the current state-of-the-art technique by 25% on both body mesh and 3D applied pressure map prediction tasks for people in bed.