Journal of Textile Research ›› 2020, Vol. 41 ›› Issue (04): 142-148.doi: 10.13475/j.fzxb.20190604207

• Apparel Engineering • Previous Articles     Next Articles

Sewing gesture recognition based on improved YOLO deep convolutional neural network

WANG Xiaohua(), YAO Weiming, WANG Wenjie, ZHANG Lei, LI Pengfei   

  1. School of Electronics and Information, Xi'an Polytechnic University, Xi'an, Shaanxi 710048, China
  • Received:2019-06-18 Revised:2020-01-12 Online:2020-04-15 Published:2020-04-27

RichHTML

6

PDF (PC)

326

Abstract:

A method of detecting and recognizing sewing gestures based on YOLO deep convolutional neural network was proposed to solve the problems of similar and less recognizable gestures in complex environments in the field of human-machine cooperation. Four complex sewing gestures were used to detect objects and construct a sewing gesture data set. By adding dense connection layer in the deep network of YOLOv3 low resolution, image feature transmission and reuse were enhanced, network performance was improved, and end-to-end sewing gesture was realized. The experimental results show that the trained model mean average precision is 94.45%, the intersection ratio is 0.87, and the harmonic mean is 0.885. By comparing region-convolutional neural networks, YOLOv2 and the original YOLOv3 algorithm, the detection accuracy of the improved method is significantly improved. At the same time, in the case of GPU acceleration, the average detection speed is 43.0 frames/s, and this fully satisfies the real-time detection of sewing gestures, and provides an algorithm basis for recognizing sewing gestures in complex environments.

Key words: sewing gesture recognition, target detection, YOLO deep convolutional neural network, garment sewing, human-machine cooperation

CLC Number: 

  • TP242.2

Fig.1

YOLO testing process"

Fig.2

YOLO improved network structure"

Tab.1

Number of images generated by data enhancement method"

手势类别 编号 原始图
像个数		 不同预处理方式下图像个数
颜色 亮度 旋转 模糊
内包缝 S1 120 120 1 100 160 100
卷边缝 S2 120 120 1 160 160 100
裁剪布料 S3 120 120 1 200 160 100
抽褶缝 S4 120 120 1 210 160 100

Tab.2

Network training parameters"

参数名 参数值
批样本大小(batch) 64个
批分割(subdivisions) 8个
迭代(iterations) 5 000次
学习率衰减步长(steps) 4 000次,4 500次
学习率衰减因子(scales) 0.1,0.1
动量(momentum) 0.9
权重衰减(decay) 0.000 5
非极大值阈值(nms) 0.25

Fig.3

Model loss curve"

Fig.4

Average cross-sectional ratio curve"

Fig.5

Average accuracy as a function of iterations number"

Fig.6

Confidence threshold"

Fig.7

Model P-R curve"

Tab.3

Experimental results of F1 at different data conditions"

数据量 500个 1 000个 2 000个 3 000个 4 000个
F1值 0.27 0.67 0.75 0.83 0.87

Tab.4

Experimental results under different lighting conditions"

时间 数据集A 数据集B
IOU值 F1值 IOU值 F1值
白天 0.858 0.806 0.874 0.862
傍晚 0.734 0.744 0.762 0.795
灯光 0.693 0.705 0.723 0.768

Fig.8

Detection effect of four sewing gestures under different lighting conditions"

Tab.5

Comparison of experimental results with different algorithms"

检测算法 mAP值/
%		 IOU值 F1值 检测速度/
(帧·s-1)
R-CNN 92.56 0.85 0.876 0.6
YOLOv2 90.04 0.76 0.865 46.0
YOLOv3 92.16 0.79 0.872 44.0
改进YOLOv3 94.45 0.87 0.885 43.0

Fig.9

Experimental results of different algorithms for four sewing gestures. (a)R-CNN;(b)YOLOv2;(c)YOLOv3;(d)Modified YOLOv3"

[1] 黄嘉俊, 柯薇, 王静, 等. 基于计算机视觉的牛仔服装色差检测评级系统[J]. 纺织学报, 2019,40(5):163-169.
HUANG Jiajun, KE Wei, WANG Jing, et al. Colorshading detection and rating system for denim based on computer vision[J]. Journal of Textile Research, 2019,40(5):163-169.
[2] 景军锋, 张星星. 基于机器视觉的玻璃纤维管纱毛羽检测[J]. 纺织学报, 2019,40(5):157-162.
JING Junfeng, ZHANG Xingxing. Fiber glass bobbin yarn hairiness detection based on machine vision[J]. Journal of Textile Research, 2019,40(5):157-162.
[3] 齐静, 徐坤, 丁希仑. 机器人视觉手势交互技术研究进展[J]. 机器人, 2017,39(4):565-584.
QI Jing, XU Kun, DING Xilun. Vision-based hand gesture recognition for human-robot interaction:a review[J]. Robot, 2017,39(4):565-584.
[4] LIU Hongyi, WANG Lihui. Gesture recognition for human-robot collaboration: a review[J]. International Journal of Industrial Ergonomics, 2017.DOI: 101016/j.ergan.2017.02.004.
pmid: 24443622
[5] ASAARI M S M, SUANDI S A, ROSDI B A. Fusion of band limited phase only correlation and width centroid contour distance for finger based biometrics[J]. Expert Systems with Applications, 2014,41(7):3367-3382.
[6] SANGQI P, MATILAINEN M, SILVEN O. Rotation tolerant hand pose recognition using aggregation of gradient orientations[M] //Lecture notes in computer science. Berlin:Springer, 2016: 257-267.
[7] MOHANTY A, RAMBHATLA S S, SAHAY R R. Deep gesture:static hand gesture recognition using CNN[M] //Advances in Intel-ligent systems and computing. Berlin:Springer, 2017: 449-461.
[8] 孟勃, 刘雪君, 王晓霖. 基于四元数时空卷积神经网络的人体行为识别[J]. 仪器仪表学报, 2017,38(11):2643-2650.
MENG Bo, LIU Xuejun, WANG Xiaolin. Human behavior recognition based on quaternion space-time convolutional neural network[J]. Chinese Journal of Scientific Instrument, 2017,38(11):2643-2650.
[9] 吴晓凤, 张江鑫, 徐欣晨. 基于Faster R-CNN的手势识别算法[J]. 计算机辅助设计与图形学学报, 2018,30(3):468-476.
WU Xiaofeng, ZHANG Jiangxin, XU Xinchen. Hand gesture recognition algorithm based on faster R-CNN[J]. Journal of Computer-Aided Design & Computer Graphics, 2018,30(3):468-476.
[10] LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector [C]//European conference on computer vision. Amsterdam:Springer, 2016: 21-37.
[11] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once:unified, real-time object detection [C]//Conference on computer vision and pattern recognition. Las Vegas: IEEE, 2016: 779-788.
[12] REDMON J, FARHADI A. YOLO9000: better, faster,stronger [C]//IEEE conference on computer vision and pattern. Hawaii: IEEE, 2017: 6517-6525.
[13] CHEN H J, WANG Q Q, YANG G W, et al. SSD object detection algorithm with multi-scale convolution feature fusion[J]. Journal of Frontiers of Computer Sci-ence and Technology, 2019,13(5):1049-1061.
[1] LIU Zhengdong, LIU Yihan, WANG Shouren. Depth learning method for suit detection in images [J]. Journal of Textile Research, 2019, 40(04): 158-164.
[2] LOU Ya-fang;ZHANG Zu-fang. Optimization of garment sewing lines scheduling based on virtual simulation [J]. JOURNAL OF TEXTILE RESEARCH, 2006, 27(6): 55-57.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
京ICP备14006648号
Copyright 2021 © Editorial office of Journal of Textile Research
Supported by Beijing Magtech Co.Ltd