Journal of Textile Research ›› 2019, Vol. 40 ›› Issue (12): 146-151.doi: 10.13475/j.fzxb.20190105306

• Comprehensive Review • Previous Articles     Next Articles

Research progress of fabric image feature extraction and retrieval based on convolutional neural network

SUN Jie1,2, DING Xiaojun1,3,4, DU Lei1,3,4, LI Qinman1, ZOU Fengyuan1,3,4()   

  1. 1. School of Fashion Design & Engineering, Zhejiang Sci-Tech University, Hangzhou, Zhejiang 310018, China
    2. School of Design & Art, Communication University of Zhejiang, Hangzhou, Zhejiang 310018, China
    3. Zhejiang Provincial Research Center of Clothing Engineering Technology, Hangzhou, Zhejiang 310018, China
    4. Key Laboratory of Silk Culture Inheriting and Products Design Technology, Ministry of Culture and Tourism, Hangzhou, Zhejiang 310018, China
  • Received:2019-01-28 Revised:2019-07-23 Online:2019-12-15 Published:2019-12-18
  • Contact: ZOU Fengyuan E-mail:zfy166@zstu.edu.cn

RichHTML

20

PDF (PC)

300

Abstract:

In order to explore the problem of automatic recognition and retrieval in fabric images, the research progress of this field in three aspects: shallow visual feature extraction, deep semantic feature representation and retrieval model construction was summarized, and then the existing problems were analyzed. The research shows that shallow visual features of fabric images have good applicability in the retrieval of small sample data sets, and the application of multi-feature fusion can effectively improve the retrieval accuracy, but there are limitations in the retrieval of large sample data sets and high-level semantic recognition. The deep convolutional neural network can effectively overcome this problem. The optimal design of fabric semantic attributes, the structural optimization of convolutional neural network and distance metric learning are three effective ways to improve the precision of deep retrieval model. It is believed that the improvement of fabric image recognition and retrieval accuracy in the future mainly depends on three aspects: accurate image segmentation and recognition technology, standardized semantic system design and multi-modal retrieval information fusion.

Key words: fabric image recognition, feature extract, image retrieval, convolutional neural network

CLC Number: 

  • TP181

Tab.1

Comparison of literatures on extraction of superficial features of fabrics"

文献 特征类别 指标/方法 样本个数 织物属性要素 是否涉及语义识别
[2] 颜色+形状特征 颜色直方图、Hu不变矩 300 纹样 否
[6] 颜色+全局特征 颜色矩、GIST特征 700 纹样 是
[8] 形状特征 离散余弦变换(DCT)特征 无 纹样 是
[13] 颜色+纹理特征 颜色直方图、LBP特征 200 风格 是
[14] 颜色+纹理特征 颜色直方图、分形纹理特征 200 表面肌理 是
[17] 纹理特征 灰度共生矩阵 无 组织结构 否
[19] 纹理特征 LBP特征 150 表面纹理 是
[22] 颜色+形态特征 三维CVQ特征、Hu不变矩 202 纹样 是
[24] 颜色+形态特征 Canny算子、颜色直方图 1 000 纹样 否

Fig.1

LeNet-5 CNN model structure"

Fig.2

Framework of fabric SBIR based on CNN"

[1] LIU Z, LUO P, QIU S, et al. Deepfashion: powering robust clothes recognition and retrieval with rich annotations[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recogintion. Seattle: IEEE Computer Society, 2016: 1096-1104.
[2] KUO C F J, LEE C, SHIH C. Image database of printed fabric with repeating dot patterns: part(I): image archiving[J]. Textile Research Journal, 2016,87(17):2089-2105.
[3] DENG D, WANG R, WU H, et al. Learning deep similarity models with focus ranking for fabric image retrieval[J]. Image and Vision Computing, 2018,70:11-20.
[4] ARUNA V, DEEPTI S. Content based image retrieval using color, texture and shape features[J]. International Journal of Advanced Research in Computer Science and Software Engineering, 2014,4(5):383-389.
[5] 丁笑君, 邹楚杭, 陈敬玉, 等. 畲族服装特征提取及其分析[J]. 纺织学报, 2015,36(7):110-115.
DING Xiaojun, ZOU Chuhang, CHEN Jingyu, et al. Extraction and classification of She nationality clothing characteristics[J]. Journal of Textile Research, 2015,36(7):110-115.
[6] JING J, LI Q, LI P, et al. A new method of printed fabric image retrieval based on color moments and gist feature description[J]. Textile Research Journal, 2015,86(11):1137-1150.
[7] HAMAD N, GUO B, NAEEM M R, et al. A new approach for image detection based on refined bag of words algorithm[J]. Optik International Journal for Light and Electron Optics, 2017,140:823-832.
doi: 10.1016/j.ijleo.2017.05.018
[8] ZHAO H, JIA G, TAN X. Fabric pattern elements retrieval based on cosine transform[J]. Computer Aided Drafting, Design and Manufacturing, 2015,25(3):18-22.
[9] KARPATH A, LI F F. Deep visual-semantic alignments for generating image descriptions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(4):664-676.
doi: 10.1109/TPAMI.2016.2598339 pmid: 27514036
[10] BENGIO Y, COURVILLE A, VINCENT P. Representation learning: a review and new perspe-ctives[J]. IEEE Transactionson Pattern Analysis and Machine Intelligence, 2013,35(8):1798-1828.
[11] DO T T, HOANG T, TAN D K L, et al. From selective deep convolutional features to compact binary representations for image retrieval[J]. Computer Vision and Pattern Recognition, 2018,1(1):1-20.
[12] FENG Q, HAO Q, CHEN Y, et al. Hybrid histogram descriptor: a fusion feature representation for image retrieval[J]. Sensors, 2018,18(6):1-22.
doi: 10.3390/s18010001
[13] CHEN Y W, HUANG X, CHEN D, et al. Generic and specific impressions estimation and their application to kansei-based clothing fabric image retrieval[J]. International Journal of Pattern Recognition and Artificial Intelligence, 2018,32(10):185-204.
[14] SUCIATI N, HERUMURTI D, WIJAYA A Y. Fractal-based texture and HSV color features for fabric image retrieval[C]// Proceedings of IEEE International Conference on Control System, Computing and engineering. Penang: IEEE Computer Society, 2015: 178-182.
[15] SZTANDERA L, CARDELLO A, WINTERHALTER C, et al. Identification of the most significant comfort factors for textiles from processing mechanical[J]. Textile Research Journal, 2013,83(1):34-43.
doi: 10.1177/0040517512438121
[16] DATTA R, JOSHI D, LI J, et al. Image retrieval: ideas, influences, and trends of the new age[J]. ACM Transactions on Computing Surveys, 2008,40(2):1-66.
[17] JING J, XU M, LI P. Automatic recognition of weave pattern and repeat for yarn-dyed fabric based on KFCM and IDMF[J]. Optik-International Journal for Light and Electron Optics, 2015,126(21):2876-2883.
doi: 10.1016/j.ijleo.2015.07.025
[18] LIU G H, YANG J Y. Image retrieval based on the texton co-occurrence matrix[J]. Pattern Recognition, 2008,41(12):3521-3527.
doi: 10.1016/j.patcog.2008.06.010
[19] YILDIZ K. Dimensionality reduction-based feature extraction and classification on fleece fabric images[J]. Signal, Image and Video Processing, 2016,11(2):317-323.
[20] YANG J, ZHAO Q, ZHOU L, et al. Research on texture images retrieval based on the gabor wavelet transform[C]// International Conference on Information Engineering. Taiyuan: IEEE Computer Society, 2009: 79-82.
[21] KUO C F J, SHIH C Y, LEE J Y. Separating color and identifying repeat pattern through the automatic computerized analysis system for printed fabrics[J]. Journal of Information Science and Engineering, 2008,24:453-467.
[22] UMAM K, BARAKBAH A R, BASUKI A. Semantic madurese batik search with cultural computing of symbolic impression extraction and analytical aggregation of color, shape and area features[J]. EMITTER International Journal of Engineering Technology, 2017,5(1):72-90.
[23] KUO C F J, SHIH C Y, LEE J Y. Repeat pattern segmentation of printed fabrics by hough transform method[J]. Textile Research Journal, 2016,75(11):779-783.
[24] 向忠, 何旋, 钱淼, 等. 基于边缘和颜色特征的织物印花花型检索[J]. 纺织学报, 2018,39(5):137-143.
XIANG Zhong, HE Xuan, QIAN Miao, et al. Printed fabric pattern retrieval based on edge and color features[J]. Journal of Textile Research, 2018,39(5):137-143.
[25] WANG C, YANG H, MEINEL C. A deep semantic framework for multimodal representation learning[J]. Multimedia Tools and Applications, 2016,75(15):9255-9276.
[26] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient based learning applied to document recognition[J]. Proceedings of IEEE, 1998,86(11):2278-2324.
[27] KRIZHEVSKY A, SUTSKEVER I, HINTON G. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017,60(6):84-90.
[28] SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE Computer Society, 2015: 1-9.
[29] WAN J, WANG D, HOI C H, et al. Deep learning for content-based image retrieval: a comprehensive study[C]// ACM International Conference on Multimedia. Florida: Association for Computing Machinery, 2014: 1809-1837.
[30] 汪珊娜, 张华熊, 康锋. 基于卷积神经网络的领带花型情感分类[J]. 纺织学报, 2018,39(8):117-132.
WANG Shanna, ZHANG Huaxiong, KANG Feng. Emotion classification of necktie pattern based on convolution neural network[J]. Journal of Textile Research, 2018,39(8):117-132.
[31] NORDIN A, MD N L, ZAINUDDIN A. Batik KR semantic network: visualizations of creative process and design knowledge for the malaysian batik designers' community[C]// Symposium on Human Interface. Berlin: Springer, 2009: 334-341.
[32] KUANG Z, YU J, YU Z, et al. Ontology-driven hierarchical deep learning for fashion recognition[C]// IEEE Conference on Multimedia Information Processing and Retrieval. Miami: IEEE Computer Society, 2018: 19-24.
[33] WANG J, SONG Y, LEUNG T, et al. Learning fine-grained image similarity with deep ranking[C]// IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE Computer Society, 2014: 1386-1393.
[34] HOFFER E, AILON N. Deep metric learning using triplet network[C]// Proceedings of International Workshop on Similarity-Based Pattern Recognition. Copenhagen: Springer International Publishing AG, 2015: 84-92.
[1] WANG Hui, SUN Jie, DING Xiaojun, LONG Ying, ZOU Fengyuan. Comparison of feature extracting and matching methods for fabric patterns [J]. Journal of Textile Research, 2020, 41(04): 45-50.
[2] WANG Xiaohua, YAO Weiming, WANG Wenjie, ZHANG Lei, LI Pengfei. Sewing gesture recognition based on improved YOLO deep convolutional neural network [J]. Journal of Textile Research, 2020, 41(04): 142-149.
[3] JIA Xiaojun, YE Lihua, DENG Hongtao, LIU Zihao, LU Fengjie. Elements classification of vein patterns using convolutional neural networks for blue calico [J]. Journal of Textile Research, 2020, 41(01): 110-117.
[4] WU Chuanbin, LIU Li, FU Xiaodong, LIU Lijun, HUANG Qingsong. Clothing image retrieval by salient region detection and sketches [J]. Journal of Textile Research, 2019, 40(07): 174-181.
[5] . Fabric defect inspection based on modified discriminant complete local binary pattern and lattice segmentation [J]. Journal of Textile Research, 2018, 39(09): 57-64.
[6] . Yarn-dyed fabric defect detection based on deep-convolutional neural network [J]. JOURNAL OF TEXTILE RESEARCH, 2017, 38(02): 68-74.
[7] . Fast fabric defect detection algorithm based on integral image [J]. JOURNAL OF TEXTILE RESEARCH, 2016, 37(11): 141-147.
[8] . Lace retrieval method based on improved texture feature [J]. JOURNAL OF TEXTILE RESEARCH, 2016, 37(06): 142-154.
[9] . Automatic recognition and positioning method of fabric slitting zone using one-dimensional Fourier transform [J]. JOURNAL OF TEXTILE RESEARCH, 2016, 37(01): 147-151.
[10] . Necktie pattern retrieval method based on image edge characteristics [J]. JOURNAL OF TEXTILE RESEARCH, 2015, 36(05): 123-126.
[11] . Algorithm for pattern detection of cotton foreign fibers based on cluster statistic analysis [J]. Journal of Textile Research, 2015, 36(03): 135-139.
[12] . Defect detection of plain weave based on visual saliency mechanism [J]. JOURNAL OF TEXTILE RESEARCH, 2014, 35(4): 56-0.
[13] . Research on image retrieval algorithm by relevance feedback technology and shape features [J]. JOURNAL OF TEXTILE RESEARCH, 2014, 35(2): 94-0.
[14] . Fabric defect detection of statistic aberration feature based on GMRF model [J]. JOURNAL OF TEXTILE RESEARCH, 2013, 34(4): 137-142.
[15] . Research of mixture feature aberrance fabric defect recognition based on self-adaptive disperse wavelet transform [J]. JOURNAL OF TEXTILE RESEARCH, 2013, 34(1): 133-137.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
京ICP备14006648号
Copyright 2021 © Editorial office of Journal of Textile Research
Supported by Beijing Magtech Co.Ltd