SMFLIX: A Unified Body-Head Model for Real-Time Expressive Human Mesh Recovery

Abstract

Expressive human mesh recovery reconstructs the body, hands, and face of a person from a single image. Whole-body models such as SMPL-X recover pose reliably but leave the face nearly static, whereas face models such as FLAME are accurate yet head-only; combining the two requires instantiating and aligning both at inference and re-annotating the training data. We present SMFLIX, a unified body–head model that merges the two bases into a single linear block-matrix formulation. Re-anchoring both bases to a common eye center aligns the head without external registration, and averaging the neck seam keeps the surface watertight. The eye and mouth components, which prior work applies to the posed mesh, are folded into the template as linear blendshape terms, so that complete blinking and jaw articulation become representable within one mesh. Since the parameters stay compatible with the existing annotations of both, we build SMFLIX-Set, 11M instances obtained by reusing and refining them, and train SMFLIX-Net, a one-stage dual-decoder network. Compared against full-body, face-only, and combined state-of-the-art methods, our network is the most accurate on every whole-body and expression metric we report, lowering the whole-mesh PA-PVE from 37.15 to 21.47 mm while recovering the facial articulation that whole-body regressors capture only coarsely. It runs at 48 FPS on a single consumer GPU, and we demonstrate this in a live webcam application.