基于卷积神经网络的织物图像特征提取与检索研究进展

doi:10.13475/j.fzxb.20190105306

纺织学报 ›› 2019, Vol. 40 ›› Issue (12): 146-151.doi: 10.13475/j.fzxb.20190105306

基于卷积神经网络的织物图像特征提取与检索研究进展

孙洁¹^,², 丁笑君¹^,³^,⁴, 杜磊¹^,³^,⁴, 李秦曼¹, 邹奉元¹^,³^,⁴()

1.浙江理工大学服装学院, 浙江杭州 310018
2.浙江传媒学院设计艺术学院, 浙江杭州 310018
3.浙江省服装工程技术研究中心, 浙江杭州 310018
4.丝绸文化传承与产品设计数字化技术文化和旅游部重点实验室, 浙江杭州 310018

收稿日期:2019-01-28 修回日期:2019-07-23 出版日期:2019-12-15 发布日期:2019-12-18
通讯作者: 邹奉元
作者简介:孙洁(1987—),女,讲师,博士生。主要研究方向为服装数字化技术。
基金资助:
浙江省教育厅项目(Y201840287);浙江省大学生科技创新项目(2018R406076);浙江省2011协同创新中心科技研发专项(17034005-F)

Research progress of fabric image feature extraction and retrieval based on convolutional neural network

SUN Jie¹^,², DING Xiaojun¹^,³^,⁴, DU Lei¹^,³^,⁴, LI Qinman¹, ZOU Fengyuan¹^,³^,⁴()

1. School of Fashion Design & Engineering, Zhejiang Sci-Tech University, Hangzhou, Zhejiang 310018, China
2. School of Design & Art, Communication University of Zhejiang, Hangzhou, Zhejiang 310018, China
3. Zhejiang Provincial Research Center of Clothing Engineering Technology, Hangzhou, Zhejiang 310018, China
4. Key Laboratory of Silk Culture Inheriting and Products Design Technology, Ministry of Culture and Tourism, Hangzhou, Zhejiang 310018, China

Received:2019-01-28 Revised:2019-07-23 Online:2019-12-15 Published:2019-12-18
Contact: ZOU Fengyuan

摘要/Abstract

摘要：

为实现织物图像的快速自动识别与检索,从织物图像浅层视觉特征提取、深度语义特征学习以及检索模型构建3个方面综述了该领域的研究进展,分析了现有研究中存在的问题。发现织物图像浅层视觉特征在小样本数据集的检索中具有较好的适用性,且多特征融合应用可有效提升检索精度,但在大样本数据集及高层语义识别检索问题中的应用存在局限性,深度卷积神经网络是克服这一问题的有效途径;织物语义属性的优化设计、卷积神经网络结构优化以及距离尺度学习是目前提升深度检索模型语义识别精度的3个有效途径;认为未来织物图像识别检索精度的提升主要依赖于标准化的语义系统设计、精准的图像分割与识别技术以及多模态的信息融合检索。

关键词: 织物图像特征, 特征提取, 图像检索, 卷积神经网络

Abstract:

In order to explore the problem of automatic recognition and retrieval in fabric images, the research progress of this field in three aspects: shallow visual feature extraction, deep semantic feature representation and retrieval model construction was summarized, and then the existing problems were analyzed. The research shows that shallow visual features of fabric images have good applicability in the retrieval of small sample data sets, and the application of multi-feature fusion can effectively improve the retrieval accuracy, but there are limitations in the retrieval of large sample data sets and high-level semantic recognition. The deep convolutional neural network can effectively overcome this problem. The optimal design of fabric semantic attributes, the structural optimization of convolutional neural network and distance metric learning are three effective ways to improve the precision of deep retrieval model. It is believed that the improvement of fabric image recognition and retrieval accuracy in the future mainly depends on three aspects: accurate image segmentation and recognition technology, standardized semantic system design and multi-modal retrieval information fusion.

Key words: fabric image recognition, feature extract, image retrieval, convolutional neural network

中图分类号:

TP181

孙洁, 丁笑君, 杜磊, 李秦曼, 邹奉元. 基于卷积神经网络的织物图像特征提取与检索研究进展[J]. 纺织学报, 2019, 40(12): 146-151.

SUN Jie, DING Xiaojun, DU Lei, LI Qinman, ZOU Fengyuan. Research progress of fabric image feature extraction and retrieval based on convolutional neural network[J]. Journal of Textile Research, 2019, 40(12): 146-151.

图/表 3

表1

图1

图2

参考文献 34

[1]	LIU Z, LUO P, QIU S, et al. Deepfashion: powering robust clothes recognition and retrieval with rich annotations[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recogintion. Seattle: IEEE Computer Society, 2016: 1096-1104.
[2]	KUO C F J, LEE C, SHIH C. Image database of printed fabric with repeating dot patterns: part(I): image archiving[J]. Textile Research Journal, 2016,87(17):2089-2105.
[3]	DENG D, WANG R, WU H, et al. Learning deep similarity models with focus ranking for fabric image retrieval[J]. Image and Vision Computing, 2018,70:11-20.
[4]	ARUNA V, DEEPTI S. Content based image retrieval using color, texture and shape features[J]. International Journal of Advanced Research in Computer Science and Software Engineering, 2014,4(5):383-389.
[5]	丁笑君, 邹楚杭, 陈敬玉, 等. 畲族服装特征提取及其分析[J]. 纺织学报, 2015,36(7):110-115.
	DING Xiaojun, ZOU Chuhang, CHEN Jingyu, et al. Extraction and classification of She nationality clothing characteristics[J]. Journal of Textile Research, 2015,36(7):110-115.
[6]	JING J, LI Q, LI P, et al. A new method of printed fabric image retrieval based on color moments and gist feature description[J]. Textile Research Journal, 2015,86(11):1137-1150.
[7]	HAMAD N, GUO B, NAEEM M R, et al. A new approach for image detection based on refined bag of words algorithm[J]. Optik International Journal for Light and Electron Optics, 2017,140:823-832. doi: 10.1016/j.ijleo.2017.05.018
[8]	ZHAO H, JIA G, TAN X. Fabric pattern elements retrieval based on cosine transform[J]. Computer Aided Drafting, Design and Manufacturing, 2015,25(3):18-22.
[9]	KARPATH A, LI F F. Deep visual-semantic alignments for generating image descriptions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(4):664-676. doi: 10.1109/TPAMI.2016.2598339 pmid: 27514036
[10]	BENGIO Y, COURVILLE A, VINCENT P. Representation learning: a review and new perspe-ctives[J]. IEEE Transactionson Pattern Analysis and Machine Intelligence, 2013,35(8):1798-1828.
[11]	DO T T, HOANG T, TAN D K L, et al. From selective deep convolutional features to compact binary representations for image retrieval[J]. Computer Vision and Pattern Recognition, 2018,1(1):1-20.
[12]	FENG Q, HAO Q, CHEN Y, et al. Hybrid histogram descriptor: a fusion feature representation for image retrieval[J]. Sensors, 2018,18(6):1-22. doi: 10.3390/s18010001
[13]	CHEN Y W, HUANG X, CHEN D, et al. Generic and specific impressions estimation and their application to kansei-based clothing fabric image retrieval[J]. International Journal of Pattern Recognition and Artificial Intelligence, 2018,32(10):185-204.
[14]	SUCIATI N, HERUMURTI D, WIJAYA A Y. Fractal-based texture and HSV color features for fabric image retrieval[C]// Proceedings of IEEE International Conference on Control System, Computing and engineering. Penang: IEEE Computer Society, 2015: 178-182.
[15]	SZTANDERA L, CARDELLO A, WINTERHALTER C, et al. Identification of the most significant comfort factors for textiles from processing mechanical[J]. Textile Research Journal, 2013,83(1):34-43. doi: 10.1177/0040517512438121
[16]	DATTA R, JOSHI D, LI J, et al. Image retrieval: ideas, influences, and trends of the new age[J]. ACM Transactions on Computing Surveys, 2008,40(2):1-66.
[17]	JING J, XU M, LI P. Automatic recognition of weave pattern and repeat for yarn-dyed fabric based on KFCM and IDMF[J]. Optik-International Journal for Light and Electron Optics, 2015,126(21):2876-2883. doi: 10.1016/j.ijleo.2015.07.025
[18]	LIU G H, YANG J Y. Image retrieval based on the texton co-occurrence matrix[J]. Pattern Recognition, 2008,41(12):3521-3527. doi: 10.1016/j.patcog.2008.06.010
[19]	YILDIZ K. Dimensionality reduction-based feature extraction and classification on fleece fabric images[J]. Signal, Image and Video Processing, 2016,11(2):317-323.
[20]	YANG J, ZHAO Q, ZHOU L, et al. Research on texture images retrieval based on the gabor wavelet transform[C]// International Conference on Information Engineering. Taiyuan: IEEE Computer Society, 2009: 79-82.
[21]	KUO C F J, SHIH C Y, LEE J Y. Separating color and identifying repeat pattern through the automatic computerized analysis system for printed fabrics[J]. Journal of Information Science and Engineering, 2008,24:453-467.
[22]	UMAM K, BARAKBAH A R, BASUKI A. Semantic madurese batik search with cultural computing of symbolic impression extraction and analytical aggregation of color, shape and area features[J]. EMITTER International Journal of Engineering Technology, 2017,5(1):72-90.
[23]	KUO C F J, SHIH C Y, LEE J Y. Repeat pattern segmentation of printed fabrics by hough transform method[J]. Textile Research Journal, 2016,75(11):779-783.
[24]	向忠, 何旋, 钱淼, 等. 基于边缘和颜色特征的织物印花花型检索[J]. 纺织学报, 2018,39(5):137-143.
	XIANG Zhong, HE Xuan, QIAN Miao, et al. Printed fabric pattern retrieval based on edge and color features[J]. Journal of Textile Research, 2018,39(5):137-143.
[25]	WANG C, YANG H, MEINEL C. A deep semantic framework for multimodal representation learning[J]. Multimedia Tools and Applications, 2016,75(15):9255-9276.
[26]	LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient based learning applied to document recognition[J]. Proceedings of IEEE, 1998,86(11):2278-2324.
[27]	KRIZHEVSKY A, SUTSKEVER I, HINTON G. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017,60(6):84-90.
[28]	SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE Computer Society, 2015: 1-9.
[29]	WAN J, WANG D, HOI C H, et al. Deep learning for content-based image retrieval: a comprehensive study[C]// ACM International Conference on Multimedia. Florida: Association for Computing Machinery, 2014: 1809-1837.
[30]	汪珊娜, 张华熊, 康锋. 基于卷积神经网络的领带花型情感分类[J]. 纺织学报, 2018,39(8):117-132.
	WANG Shanna, ZHANG Huaxiong, KANG Feng. Emotion classification of necktie pattern based on convolution neural network[J]. Journal of Textile Research, 2018,39(8):117-132.
[31]	NORDIN A, MD N L, ZAINUDDIN A. Batik KR semantic network: visualizations of creative process and design knowledge for the malaysian batik designers' community[C]// Symposium on Human Interface. Berlin: Springer, 2009: 334-341.
[32]	KUANG Z, YU J, YU Z, et al. Ontology-driven hierarchical deep learning for fashion recognition[C]// IEEE Conference on Multimedia Information Processing and Retrieval. Miami: IEEE Computer Society, 2018: 19-24.
[33]	WANG J, SONG Y, LEUNG T, et al. Learning fine-grained image similarity with deep ranking[C]// IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE Computer Society, 2014: 1386-1393.
[34]	HOFFER E, AILON N. Deep metric learning using triplet network[C]// Proceedings of International Workshop on Similarity-Based Pattern Recognition. Copenhagen: Springer International Publishing AG, 2015: 84-92.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

文献	特征类别	指标/方法	样本个数	织物属性要素	是否涉及语义识别
[2]	颜色+形状特征	颜色直方图、Hu不变矩	300	纹样	否
[6]	颜色+全局特征	颜色矩、GIST特征	700	纹样	是
[8]	形状特征	离散余弦变换(DCT)特征	无	纹样	是
[13]	颜色+纹理特征	颜色直方图、LBP特征	200	风格	是
[14]	颜色+纹理特征	颜色直方图、分形纹理特征	200	表面肌理	是
[17]	纹理特征	灰度共生矩阵	无	组织结构	否
[19]	纹理特征	LBP特征	150	表面纹理	是
[22]	颜色+形态特征	三维CVQ特征、Hu不变矩	202	纹样	是
[24]	颜色+形态特征	Canny算子、颜色直方图	1 000	纹样	否

基于卷积神经网络的织物图像特征提取与检索研究进展

Research progress of fabric image feature extraction and retrieval based on convolutional neural network

RichHTML

PDF (PC)

摘要/Abstract

引用本文

使用本文

图/表 3

参考文献 34

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	邵金鑫, 张宝昌, 曹继鹏. 基于图像处理与深度学习方法的棉纤维梳理过程纤维检测识别技术[J]. 纺织学报, 2020, 41(07): 40-46.
[2]	王泽霞, 陈革, 陈振中. 基于改进卷积神经网络的化纤丝饼表面缺陷识别[J]. 纺织学报, 2020, 41(04): 39-44.
[3]	汪会, 孙洁, 丁笑君, 龙颖, 邹奉元. 织物纹样特征提取与匹配方法比较[J]. 纺织学报, 2020, 41(04): 45-50.
[4]	王晓华, 姚炜铭, 王文杰, 张蕾, 李鹏飞. 基于改进YOLO 深度卷积神经网络的缝纫手势检测[J]. 纺织学报, 2020, 41(04): 142-149.
[5]	贾小军, 叶利华, 邓洪涛, 刘子豪, 陆锋杰. 基于卷积神经网络的蓝印花布纹样基元分类 [J]. 纺织学报, 2020, 41(01): 110-117.
[6]	吴传彬, 刘骊, 付晓东, 刘利军, 黄青松. 结合显著区域检测和手绘草图的服装图像检索[J]. 纺织学报, 2019, 40(07): 174-181.
[7]	刘正东, 刘以涵, 王首人. 西装识别的深度学习方法[J]. 纺织学报, 2019, 40(04): 158-164.
[8]	吴欢, 丁笑君, 李秦曼, 杜磊, 邹奉元. 采用卷积神经网络 CaffeNet 模型的女裤廓形分类[J]. 纺织学报, 2019, 40(04): 117-121.
[9]	赵树志狄岚何锐波. 基于改进判别性完整局部二值模式与格分割的织物瑕疵检测方法[J]. 纺织学报, 2018, 39(09): 57-64.
[10]	汪珊娜张华熊康锋. 基于卷积神经网络的领带花型情感分类[J]. 纺织学报, 2018, 39(08): 117-123.
[11]	王雯雯高畅刘基宏. 应用卷积神经网络的细纱断纱锭位识别[J]. 纺织学报, 2018, 39(06): 136-141.
[12]	向忠何旋钱淼胡旭东. 基于边缘和颜色特征的织物印花花型检索[J]. 纺织学报, 2018, 39(05): 137-143.
[13]	景军锋范晓婷李鹏飞洪良. 应用深度卷积神经网络的色织物缺陷检测[J]. 纺织学报, 2017, 38(02): 68-74.
[14]	董蓉李勃徐晨. 应用积分图的织物瑕疵检测快速算法[J]. 纺织学报, 2016, 37(11): 141-147.
[15]	曹霞李岳阳罗海驰蒋高明丛洪莲. 蕾丝花边的改进型纹理特征检索方法[J]. 纺织学报, 2016, 37(06): 142-154.