Abstract:

Objective In virtual try-on and garment customization, the traditional 3-D human reconstruction method faces limitations, where the single-view methods yield inaccuracies due to depth ambiguity/occlusion while multi-view approaches ignore topological correlations. This study aims to overcome these by innovatively integrating CNNs and GCNs to develop an accurate video-frame-to-3-D-body mapping, and to establish a framework capable of solving nonlinear multi-view feature aggregation which enables efficient reconstruction using consumer-grade video inputs instead of professional scanners.

Method This study proposes a novel 3-D human body reconstruction framework that integrates Convolutional Neural Networks (CNN) with Graph Convolutional Neural Networks (GCN). To address the challenge of sparse view reconstruction, multi-view binary masks are extracted from a 360° rotating video and the 3-D human shape is compressed into a 25-dimensional PCA parameter space. The core architecture employs a modified ResNet-50 embedded with 3-D convolutional layers and Convolutional Block Attention Modules (CBAM) to capture spatiotemporal features and enhance structural discriminability. Furthermore, a two-layer GCN is utilized to model the spatial correlations between different viewpoints via an adjacency matrix. By modeling the multi-view inputs as nodes in a graph structure, the GCN dynamically aggregates spatial context, enabling the network to learn the geometric relationship between viewpoints and resolve occlusion ambiguities inherent in individual projections. This integration enables precise regression of PCA coefficients for high-fidelity mesh generation. Experimental results on the SPRING dataset demonstrate that the proposed method effectively recovers complex body geometries with high computational efficiency and robustness.

Results The proposed method leads to significant improvements in both accuracy and efficiency for 3-D human body reconstruction. The proposed method achieves a Chamfer Distance (CD) of 1.12 cm, outperforming existing methods. Furthermore, the reconstruction precision is demonstrated by an average per-vertex error of less than 0.5 cm, with relative errors for all primary human body parameters maintained below 5%, confirming the high geometric fidelity of the reconstructed model. The ablation study confirmed the critical contribution of the GCN module: the exclusion of this component resulted in a significant increase in both the average per-vertex error and the maximum error, demonstrating the module's effectiveness in capturing complex inter-view relationships. Visual comparisons between reconstructed models and ground-truth meshes validated the method's capability for recovering fine-grained anatomical details. Practical utility was demonstrated in apparel customization scenarios, where virtual try-on applications leveraging the precise body models significantly reduced costs associated with physical garment trials. Although the GCN component introduced additional computational overhead during training, the achieved reconstruction quality substantially surpassed traditional CNN cascade strategies.

Conclusion This study proposes an innovative CNN-GCN fusion framework that effectively addresses the core challenge of insufficient view-correlation modeling in multi-view human body reconstruction. By integrating locally extracted CNN features with topological relationships among views captured by the GCN, the approach significantly enhances reconstruction accuracy. The method provides an efficient solution for virtual try-on and personalized garment customization scenarios, achieving high-precision modeling using only video captured by ordinary cameras, thereby substantially reducing the cost barrier associated with traditional 3-D scanning equipment. However, training efficiency requires further optimization due to computational overhead from sparse matrix operations in the GCN. Future work will focus on developing lightweight graph network architectures to accelerate inference while extending the framework to dynamic reconstruction applications.

Key words: 3-D human reconstruction, parametric human model, graph neural network, attention mechanism