Abstract:

Objective The objective of this study is to develop a deep-learning-based real-time detection model for clothing keypoints, aiming to address the challenge of balancing accuracy and real-time performance in complex scenarios. The research focuses on enhancing the robustness and precision of keypoint detection across various clothing types. This is essential for advancing applications in intelligent clothing manufacturing and virtual fitting. Existing models struggle with occlusions, diverse clothing styles, varying keypoint sizes, and real-time performance. This work aims to overcome these limitations while maintaining high computational efficiency.

Method This study proposes the Real-Time Fashion Pose Estimation (RTFPose) model for real-time clothing keypoint detection, based on the RTMPose architecture. RTFPose includes the Median Enhanced Channel and Spatial Attention Module (MECS) to enhance key area features and reduce noise for better occlusion detection. The Cross-Scale Feature Fusion Module (CSFF) integrates multi-scale features to handle varying keypoint sizes. The Self-Attention Feature Enhancement Module (SAFE) focuses on keypoint regions to suppress background interference. Additionally, a finetuning strategy addresses data imbalance.

Results The RTFPose model demonstrated excellent performance on the DeepFashion2 dataset, which achieved a high speed of 140 frames/s with an Area Under the Curve (AUC) value of 65.1%, a significant 6.5% improvement in accuracy compared to the baseline model RTMPose. Additionally, on the DeepFashion dataset, the model achieved the percentage of correct keypoints (P) of 68.0% at a real-time speed of 142.3 frames/s. These results further demonstrate the model's strong performance in keypoint detection accuracy while maintaining high efficiency and validate its good generalization capability across different datasets. The model shows that the introduction of the MECS separately improves the accuracy to 59.5%. Through the channel-spatial attention mechanism, it effectively improves the feature visibility of clothing keypoints in occluded scenes. After stacking the CSFF, the accuracy further increased to 62.0%. This module integrates multi-level features of the backbone network and solves the problem of keypoint size variability by fusing high and low-resolution details and semantic information. After further introducing the SAFE, the performance reached 63.8%. The self-attention mechanism adaptively focused on keypoint areas, reduced background texture interference (such as clothing folds and decorative patterns), and improved feature purity. The final overlay classification fine-tuning strategy achieved a model accuracy of 65.1%. The fine-tuning was achieved by independently training six types of clothing, balancing the accuracy of keypoint detection for each type of clothing. These results highlight the effectiveness of the proposed modules and the fine-tuning strategy in enhancing the robustness and accuracy of the RTFPose model in complex scenarios. The model's ability to maintain high efficiency while improving detection accuracy makes it a valuable solution for real-time clothing key-point detection in various industrial applications.

Conclusion In conclusion, the proposed model effectively balances real-time performance and detection accuracy for clothing keypoint detection. By integrating MECS, CSFF, and SAFE, the model significantly enhances its robustness and accuracy in complex scenarios. Additionally, the fine-tuning strategy effectively addresses data imbalance, improving detection performance across different clothing types. The lightweight design and high efficiency of the proposed model make it particularly valuable for industrial applications such as smart clothing manufacturing and virtual fitting. Future work will focus on three main directions: firstly enhancing the system's adaptability to dynamic scenes to improve robustness and real-time processing capabilities in dynamic environments; secondly, utilizing multimodal data fusion technology to integrate depth information and texture features, thereby improving recognition accuracy; thirdly, adopting a self-supervised learning paradigm to reduce dependence on manual annotation and enhance the model's generalization performance. These advancements will further strengthen the applicability and effectiveness of the proposed model in various industrial settings.

Key words: keypoint of clothing, real-time detection, deep learning, attention enhancement mechanism, cross-scale feature fusion