SS-MVMETRO: Semi-supervised multi-view human mesh recovery transformer.

Parametric methods are widely utilized in RGB-based human mesh recovery, relying on precise statistical human body model parameters that are challenging to obtain. In contrast, non-parametric transformer-based approaches excel but are applied only to monocular RGB tasks. To address these limitations, this paper presents Semi-Supervised Multi-View Human Mesh Recovery Transformer (SS-MVMETRO), which combines multi-view information with non-parametric methods for the first time. Our model encodes different images according to their respective view directions, fusing local features around key points of joints and vertices. Then, a residual-like structure is proposed to integrate the fused features in the mesh recovery transformer, which subsequently predicts the 3D coordinates of the human mesh vertices. Additionally, we divide different views into the main view and auxiliary views and propose a semi-supervised training approach that requires fewer matching labels. The efficacy of our work is validated on two datasets, Human3.6M and Mpi_inf_3dph, through quantitative and qualitative experiments. The results indicate that SS-MVMETRO improves the reconstruction accuracy, surpassing previous image-based methods by at least 8.9% in terms of Procrustes Analysis Mean-Per-Joint-Position-Error (PA-MPJPE). [ABSTRACT FROM AUTHOR]