基于計算機視覺(jué)的工業(yè)金屬表面缺陷檢測綜述

伍麟; 郝鴻宇; 宋友

doi:10.16383/j.aas.c230039

基于計算機視覺(jué)的工業(yè)金屬表面缺陷檢測綜述

doi: 10.16383/j.aas.c230039

伍麟^1,,
郝鴻宇^1,,
宋友^1,

1.
北京航空航天大學(xué)軟件學(xué)院北京 100191

詳細信息

作者簡(jiǎn)介:
伍麟：北京航空航天大學(xué)碩士研究生. 主要研究方向為計算機視覺(jué), 目標檢測和表面缺陷檢測. E-mail: zf2021349@buaa.edu.cn

郝鴻宇：北京航空航天大學(xué)碩士研究生. 主要研究方向為計算機視覺(jué), 圖神經(jīng)網(wǎng)絡(luò )和少樣本學(xué)習. E-mail: JoeyHao@buaa.edu.cn

宋友：北京航空航天大學(xué)教授. 主要研究方向為軟件工程, 異常信號檢測, 算法分析與設計. 本文通信作者. E-mail: songyou@buaa.edu.cn

計量
- 文章訪(fǎng)問(wèn)數: 2997
- HTML全文瀏覽量: 3513
- PDF下載量: 828
- 被引次數: 0
出版歷程
- 收稿日期: 2023-02-06
- 錄用日期: 2023-05-18
- 網(wǎng)絡(luò )出版日期: 2023-07-03
- 刊出日期: 2024-07-23

A Review of Metal Surface Defect Detection Based on Computer Vision

WU Lin^1
,,
HAO Hong-Yu^1
,,
SONG You^1
,

1.
School of Software, Beihang University, Beijing 100191

More Information

Author Bio:
WU Lin　Master student at the School of Software, Beihang University. His research interest covers computer vision, object detection and surface defect detection

HAO Hong-Yu　Master student at the School of Software, Beihang University. His research interest covers computer vision, graph neural network and few-shot learning

SONG You　Professor at the School of Software, Beihang University. His research interest covers software engineering, anomaly signal detection, algorithm analysis and design. Corresponding author of this paper

摘要

摘要: 針對平面及三維結構金屬材料的工業(yè)表面缺陷檢測, 概述了視覺(jué)檢測技術(shù)的基本原理和研究現狀, 并總結出視覺(jué)自動(dòng)檢測系統的關(guān)鍵技術(shù)包括光學(xué)成像技術(shù)、圖像預處理技術(shù)與缺陷檢測器. 首先介紹了如何根據檢測對象的光學(xué)特性選擇合適的二維、三維光學(xué)成像技術(shù); 其次介紹了圖像降噪、特征提取、圖像分割和拼接等預處理技術(shù)的重要作用; 然后根據缺陷檢測器的實(shí)現原理將其分為模板匹配、圖像分類(lèi)、圖像語(yǔ)義分割、目標檢測和圖像異常檢測五類(lèi), 并對其中的經(jīng)典算法進(jìn)行了歸納分析. 最后, 探討了工業(yè)場(chǎng)景下金屬表面缺陷檢測技術(shù)實(shí)施中的關(guān)鍵問(wèn)題, 并對該技術(shù)的發(fā)展趨勢進(jìn)行了展望.
- 表面缺陷檢測 /
- 計算機視覺(jué) /
- 金屬表面缺陷 /
- 自動(dòng)化檢測
Abstract: Focusing on the industrial surface defect detection of metal planar and three-dimensional structural materials, this paper summarizes the basic principle and research status of visual defect detection technology, and summarizes the key technologies of visual automatic detection system including optical imaging technology, image preprocessing technology and defect detector. Firstly, this paper introduces how to select suitable 2D and 3D optical imaging technology according to the optical characteristics of the test object. Secondly, the important functions of image denoising, feature extraction, image segmentation and image Mosaic are introduced. Then, according to the implementation principle of defect detector, it is divided into five categories: Template matching, image classification, image semantic segmentation, target detection and image anomaly detection, and the classical algorithms are summarized and analyzed. Finally, this paper discusses the key problems in the implementation of surface defect detection in the industrial scene, and looks forward to the development trend of this technology.
- Surface defect detection /
- computer vision /
- metal surface defect /
- automatic inspection system

HTML全文

圖 1 金屬表面缺陷檢測基本流程

Fig. 1 Pipline of metal surface defect detection

下載: 全尺寸圖片幻燈片

圖 2 自動(dòng)光學(xué)成像系統

Fig. 2 Automated optical inspection system

下載: 全尺寸圖片幻燈片

圖 3 表面散射模型

Fig. 3 Light scattering model on surface

下載: 全尺寸圖片幻燈片

圖 4 照明光路類(lèi)型

Fig. 4 Types of lighting path

下載: 全尺寸圖片幻燈片

圖 5 二維成像與三維成像對比

Fig. 5 2D imaging versus 3D imaging

下載: 全尺寸圖片幻燈片

圖 6 光度立體法

Fig. 6 Photometric stereo

下載: 全尺寸圖片幻燈片

圖 7 結構光法

Fig. 7 Structured light illumination

下載: 全尺寸圖片幻燈片

圖 8 混合成像技術(shù) ((a) ~ (c) 二維灰度圖像; (d) ~ (f)具有三維深度信息表示的圖像)

Fig. 8 Hybrid imaging technique ((a) ~ (c) 2D grayscale images; (d) ~ (f) Images with 3D depth information represented)

下載: 全尺寸圖片幻燈片

圖 9 基于圖像分割的缺陷檢測

Fig. 9 Defect detection based on image segmentation

下載: 全尺寸圖片幻燈片

圖 10 三元網(wǎng)絡(luò )結構

Fig. 10 The structure of triplet network

下載: 全尺寸圖片幻燈片

圖 11 二階段網(wǎng)絡(luò )和一階段網(wǎng)絡(luò )對比

Fig. 11 Comparison of two-stage and one-stage networks

下載: 全尺寸圖片幻燈片

圖 12 基于二階段網(wǎng)絡(luò )的金屬表面缺陷檢測^[100], 經(jīng)許可轉載自文獻[100], ?Sage, 2021

Fig. 12 Metal surface defect detection based on two-stage networks^[100], reproduced with permission from reference [100], ?Sage, 2021

下載: 全尺寸圖片幻燈片

圖 13 DETR網(wǎng)絡(luò )結構

Fig. 13 The network architecture of DETR

下載: 全尺寸圖片幻燈片

圖 14 基于圖像重建的缺陷檢測 ((a)變分自編碼機; (b) GAN結合AE; (c)基于記憶池的模型)

Fig. 14 Defect detection based on image reconstruction ((a) VAE; (b) GAN associated with Auto-Encoder; (c) Memory based model)

下載: 全尺寸圖片幻燈片

圖 15 基于標準化流的缺陷檢測 ((a)原始圖像; (b)多尺度輸入; (c)圖像特征分布; (d)簡(jiǎn)單分布; (e)標準分布; (f)異常分布)

Fig. 15 Defect detection based on normalizing flow ((a) Origin image; (b) Multiscale input; (c) Feature distribution; (d) Simple distribution; (e) Normalized distribution; (f) Anomalous distribution)

下載: 全尺寸圖片幻燈片

圖 16 (a)基于教師?學(xué)生網(wǎng)絡(luò )的方法; (b)基于最典型嵌入表示的方法

Fig. 16 (a) Method based on teacher-student network; (b) Method based on the most typical embedding representation

下載: 全尺寸圖片幻燈片

表 1 目標檢測模型在NEU-DET上的表現

Table 1 Performance of object detection models on NEU-DET

Method	Backbone	Neck	$AP_{50}$
Faster R-CNN^[97]	ResNet-50	FPN	74.7
Cascade R-CNN^[99]	ResNet-50	FPN	75.8
YOLOX^[115]	CSPDarknet	PA-FPN	70.9
YOLOv4^[105]	CSPDarknet	FPN	76.4
AutoAssign^[116]	ResNet-50	FPN	76.6
AutoAssign^[116]	Swin-Tiny	FPN	78.3
DDN^[108]	ResNet-50	MFN	82.3
CA-AutoAssign^[117]	CSPDarknet	CA	82.7

下載: 導出CSV

表 2 異常檢測方法對比

Table 2 Comparison of abnormal detection

Method	Detection AUROC	Segmentation AUROC	FPS
PatchCore Large^[145]	99.6	98.2	5.9
PNI^[146]	99.5	99.0	—
MemSeg^[144]	99.5	99.6	31.3
Fastflow^[141]	99.4	98.5	21.8
EfficientAD-M^[148]	99.1	96.9	269.0
EfficientAD-S^[148]	98.8	96.8	614.0
CS-Flow^[140]	98.7	—	—
Patch SVDD^[142]	92.1	95.7	2.1
VAE-Grad^[136]	89.2	—	—

下載: 導出CSV

表 3 缺陷檢測方法對比

Table 3 Comparison of defect detection methods

方法	基本原理	應用場(chǎng)景	優(yōu)缺點(diǎn)
模板匹配	比較模板與待檢樣本的差異來(lái)判斷是否存在缺陷	產(chǎn)品高度一致的金屬精密加工制成品, 例如手機外殼、汽車(chē)零件等	方法簡(jiǎn)單有效, 但需要提取制作模板, 僅適用于一致性強的產(chǎn)品
分類(lèi)網(wǎng)絡(luò )	直接用 CNN 提取特征, 通過(guò) Softmax 或距離度量來(lái)預測類(lèi)別	公差較大、尺寸較小的金屬制品, 例如螺母、金屬蓋等零件	結構簡(jiǎn)單, 是其他網(wǎng)絡(luò )的基礎, 準確率依賴(lài)缺陷樣本數量, 難以定位缺陷位置
目標檢測	對每個(gè)提議候選框或者每個(gè)網(wǎng)格進(jìn)行密集預測, 從背景中找出所有目標的分類(lèi)和位置	適用于絕大多數缺陷類(lèi)別可事先定義的工業(yè)場(chǎng)景	速度快, 適用范圍廣, 但網(wǎng)絡(luò )結構復雜, 依賴(lài)大量缺陷樣本進(jìn)行訓練
語(yǔ)義分割	通過(guò)卷積提取高階語(yǔ)義特征, 然后通過(guò)上采樣輸出像素級的缺陷邊界劃分	大面積金屬板、帶制品, 缺陷具有成片連續區域、形態(tài)不定的場(chǎng)景	可以進(jìn)行像素級缺陷分割, 但是依賴(lài)大量像素級標注數據, 標注成本很高
異常檢測	通過(guò)自編碼機、GAN、標準流等生成模型學(xué)習正常樣本的表達方式, 根據重建誤差、梯度或分布差異來(lái)進(jìn)行缺陷檢測	缺乏缺陷樣本, 只有正常樣本可以用于訓練的場(chǎng)景	無(wú)需缺陷樣本和標注, 可以檢測未事先定義的缺陷類(lèi)別, 但準確率尚達不到有監督學(xué)習的效果

下載: 導出CSV

<button id="qm3rj"><thead id="qm3rj"></thead></button>

亚洲第一网址_国产国产人精品视频69_久久久久精品视频_国产精品第九页

參考文獻(148)

[1]	Neogi N, Mohanta D K, Dutta P K. Review of vision-based steel surface inspection systems. EURASIP Journal on Image and Video Processing, 2014, 2014(1): Article No. 50 doi: 10.1186/1687-5281-2014-50
[2]	盧榮勝, 吳昂, 張騰達, 王永紅. 自動(dòng)光學(xué)(視覺(jué))檢測技術(shù)及其在缺陷檢測中的應用綜述. 光學(xué)學(xué)報, 2018, 38(8): Article No. 0815002 Lu Rong-Sheng, Wu Ang, Zhang Teng-Da, Wang Yong-Hong. Review on automated optical (visual) inspection and its applications in defect detection. Acta Optica Sinica, 2018, 38(8): Article No. 0815002
[3]	呂承侃, 沈飛, 張正濤, 張峰. 圖像異常檢測研究現狀綜述. 自動(dòng)化學(xué)報, 2022, 48(6): 1402?1428 Lv Cheng-Kan, Shen Fei, Zhang Zheng-Tao, Zhang Feng. Review of image anomaly detection. Acta Automatica Sinica, 2022, 48(6): 1402?1428
[4]	李維創(chuàng ), 尹柏強. 工業(yè)金屬板帶材表面缺陷自動(dòng)視覺(jué)檢測研究進(jìn)展. 電子測量與儀器學(xué)報, 2021, 35(6): 1?16 doi: 10.13382/j.jemi.B2003349 Li Wei-Chuang, Yin Bai-Qiang. Research progress of automated visual surface defect detection for industrial metal planar materials. Journal of Electronic Measurement and Instrumentation, 2021, 35(6): 1?16 doi: 10.13382/j.jemi.B2003349
[5]	陶顯, 侯偉, 徐德. 基于深度學(xué)習的表面缺陷檢測方法綜述. 自動(dòng)化學(xué)報, 2021, 47(5): 1017?1034 Tao Xian, Hou Wei, Xu De. A survey of surface defect detection methods based on deep learning. Acta Automatica Sinica, 2021, 47(5): 1017?1034
[6]	羅東亮, 蔡雨萱, 楊子豪, 章哲彥, 周瑜, 白翔. 工業(yè)缺陷檢測深度學(xué)習方法綜述. 中國科學(xué): 信息科學(xué), 2022, 52(6): 1002?1039 doi: 10.1360/SSI-2021-0336 Luo Dong-Liang, Cai Yu-Xuan, Yang Zi-Hao, Zhang Zhe-Yan, Zhou Yu, Bai Xiang. Survey on industrial defect detection with deep learning. Scientia Sinica Informationis, 2022, 52(6): 1002?1039 doi: 10.1360/SSI-2021-0336
[7]	Khan S, Naseer M, Hayat M, Zamir S W, Khan F S, Shah M. Transformers in vision: A survey. ACM Computing Surveys, 2022, 54(10s): Article No. 200
[8]	Huang Y C, Hung K C, Liu C C, Chuang T H, Chiou S J. Customized convolutional neural networks technology for machined product inspection. Applied Sciences, 2022, 12(6): Article No. 3014 doi: 10.3390/app12063014
[9]	Liu S, Wang Q, Luo Y. A review of applications of visual inspection technology based on image processing in the railway industry. Transportation Safety and Environment, 2019, 1(3): 185?204
[10]	Lu R S, Forrest A K. 3D surface topography from the specular lobe of scattered light. Optics and Lasers in Engineering, 2007, 45(10): 1018?1027 doi: 10.1016/j.optlaseng.2007.04.008
[11]	Smith B. Geometrical shadowing of a random rough surface. IEEE Transactions on Antennas and Propagation, 1967, 15(5): 668?671 doi: 10.1109/TAP.1967.1138991
[12]	Garcia-Lamont F, Cervantes J, López A, Rodriguez L. Segmentation of images by color features: A survey. Neurocomputing, 2018, 292: 1?27 doi: 10.1016/j.neucom.2018.01.091
[13]	Foucher B. Infrared machine vision: A new contender. In: Proceedings of the SPIE 3700, Thermosense XXI. Orlando, USA: SPIE, 1999. 210?213
[14]	Nayar S K, Ikeuchi K, Kanade T. Surface reflection: Physical and geometrical perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1991, 13(7): 611?634 doi: 10.1109/34.85654
[15]	Porter T F, Sylvester R A, Bouyoucas T W, Kolesar M P. Automatic strip surface defect detection system. Iron and Steel Engineer, 1988, 65(12): 17?20
[16]	Tian G Y, Lu R S, Gledhill D. Surface measurement using active vision and light scattering. Optics and Lasers in Engineering, 2007, 45(1): 131?139 doi: 10.1016/j.optlaseng.2006.03.005
[17]	盧榮勝. 自動(dòng)光學(xué)檢測技術(shù)的發(fā)展現狀. 紅外與激光工程, 2008, 37(S1): 120?123 Lu Rong-Sheng. State of the art of automated optical inspection. Infrared and Laser Engineering, 2008, 37(S1): 120?123
[18]	Rinn R, Thompson S A, Foehr R, Luecking F, Torre J. Parsytec HTS-2: Defect detection and classification through software vs. dedicated hardware. In: Proceedings of the Electronic Imaging '99. San Jose, CA, United States: SPIE, 1999. 110?121
[19]	李松, 周亞同, 張忠偉, 池越, 韓春穎. 基于雙打光模板匹配的沖壓件表面缺陷檢測. 鍛壓技術(shù), 2018, 43(11): 137?145 Li Song, Zhou Ya-Tong, Zhang Zhong-Wei, Chi Yue, Han Chun-Ying. Surface defect detection of stamping parts based on double light pattern matching. Forging & Stamping Technology, 2018, 43(11): 137?145
[20]	Landstrom A, Thurley M J. Morphology-based crack detection for steel slabs. IEEE Journal of Selected Topics in Signal Processing, 2012, 6(7): 866?875 doi: 10.1109/JSTSP.2012.2212416
[21]	Woodham R J. Photometric method for determining surface orientation from multiple images. Optical Engineering, 1980, 19(1): Article No. 191139
[22]	Berthold K P H. Shape From Shading: A Method for Obtaining the Shape of a Smooth Opaque Object from One View, Technical Report AITR-232, Artificial Intelligence Laboratory, MIT, USA, 1970.
[23]	Wang L, Xu K, Zhou P. Online detection technique of 3D defects for steel strips based on photometric stereo. In: Proceedings of the 8th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA). Macao, China: IEEE, 2016. 428?432
[24]	Delpy D T, Cope M, van der Zee P, Arridge S, Wray S, Wyatt J. Estimation of optical pathlength through tissue from direct time of flight measurement. Physics in Medicine & Biology, 1988, 33(12): 1433?1442
[25]	Bologna F, Tannous M, Romano D, Stefanini C. Automatic welding imperfections detection in a smart factory via 2-D laser scanner. Journal of Manufacturing Processes, 2022, 73: 948?960 doi: 10.1016/j.jmapro.2021.10.046
[26]	Huang C, Wang G L, Song H, Li R S, Zhang H O. Rapid surface defects detection in wire and arc additive manufacturing based on laser profilometer. Measurement, 2022, 189: Article No. 110503 doi: 10.1016/j.measurement.2021.110503
[27]	Li J L, Liu T, Wang X F. Advanced pavement distress recognition and 3D reconstruction by using GA-DenseNet and binocular stereo vision. Measurement, 2022, 201: Article No. 111760 doi: 10.1016/j.measurement.2022.111760
[28]	Li B Z, Xu Z J, Gao F, Cao Y L, Dong Q C. 3D reconstruction of high reflective welding surface based on binocular structured light stereo vision. Machines, 2022, 10(2): Article No. 159 doi: 10.3390/machines10020159
[29]	Zhou P, Xu K, Wang D D. Rail profile measurement based on line-structured light vision. IEEE Access, 2018, 6: 16423?16431 doi: 10.1109/ACCESS.2018.2813319
[30]	Guillo L, Jiang X R, Lafruit G, Guillemot C. Light Field Video Dataset Captured by A R8 Raytrix Camera (With Disparity Maps), Technical Report hal-01804578, International Organisation for Standardisation, San Diego, USA, 2018.
[31]	Saiz F A, Barandiaran I, Arbelaiz A, Gra?a M. Photometric stereo-based defect detection system for steel components manufacturing using a deep segmentation network. Sensors, 2022, 22(3): Article No. 882 doi: 10.3390/s22030882
[32]	Wen X, Song K C, Huang L M, Niu M H, Yan Y H. Complex surface ROI detection for steel plate fusing the gray image and 3D depth information. Optik, 2019, 198: Article No. 163313 doi: 10.1016/j.ijleo.2019.163313
[33]	Niu M H, Song K C, Huang L M, Wang Q, Yan Y H, Meng Q G. Unsupervised saliency detection of rail surface defects using stereoscopic images. IEEE Transactions on Industrial Informatics, 2021, 17(3): 2271?2281
[34]	Ren Z H, Fang F Z, Yan N, Wu Y. State of the art in defect detection based on machine vision. International Journal of Precision Engineering and Manufacturing-Green Technology, 2022, 9(2): 661?691 doi: 10.1007/s40684-021-00343-6
[35]	Niblack W. An Introduction to Digital Image Processing. Birkeroed: Strandberg Publishing Company, 1985.
[36]	Luisier F, Blu T, Unser M. A new SURE approach to image denoising: Interscale orthonormal wavelet thresholding. IEEE Transactions on Image Processing, 2007, 16(3): 593?606 doi: 10.1109/TIP.2007.891064
[37]	Gu S H, Zhang L, Zuo W M, Feng X C. Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014. 2862?2869
[38]	Mohanaiah P, Sathyanarayana P, GuruKumar L. Image texture feature extraction using GLCM approach. International Journal of Scientific and Research Publications, 2013, 3(5): 1?5
[39]	Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7): 971?987 doi: 10.1109/TPAMI.2002.1017623
[40]	Dalal N, Triggs B. Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). San Diego, USA: IEEE, 2005. 886?893
[41]	Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91?110 doi: 10.1023/B:VISI.0000029664.99615.94
[42]	Lindeberg T. Scale invariant feature transform. Scholarpedia, 2012, 7(5): Article No. 10491 doi: 10.4249/scholarpedia.10491
[43]	Oren M, Papageorgiou C, Sinha P, Osuna E, Poggio T. Pedestrian detection using wavelet templates. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Juan, USA: IEEE, 1997. 193?199
[44]	Lienhart R, Maydt J. An extended set of Haar-like features for rapid object detection. In: Proceedings of the International Conference on Image Processing. Rochester, USA: IEEE, 2002.
[45]	Wang X W, Ding X Q, Liu C S. Gabor filters-based feature extraction for character recognition. Pattern Recognition, 2005, 38(3): 369?379 doi: 10.1016/j.patcog.2004.08.004
[46]	Raheja J L, Kumar S, Chaudhary A. Fabric defect detection based on GLCM and Gabor filter: A comparison. Optik, 2013, 124(23): 6469?6474 doi: 10.1016/j.ijleo.2013.05.004
[47]	Siew L H, Hodgson R M, Wood E J. Texture measures for carpet wear assessment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1988, 10(1): 92?105 doi: 10.1109/34.3870
[48]	Liu K, Wang H Y, Chen H Y, Qu E Q, Tian Y, Sun H X. Steel surface defect detection using a new Haar–Weibull-Variance model in unsupervised manner. IEEE Transactions on Instrumentation and Measurement, 2017, 66(10): 2585?2596 doi: 10.1109/TIM.2017.2712838
[49]	Pernkopf F, O'Leary P. Image acquisition techniques for automatic visual inspection of metallic surfaces. NDT & E International, 2003, 36(8): 609?617
[50]	Kaggle. Severstal: Steel defect detection [Online], available: https://www.kaggle.com/c/severstal-steel-defect-detection, March 19, 2023
[51]	Qi S, Yang J, Zhong Z. A review on industrial surface defect detection based on deep learning technology. In: Proceedings of the 3rd International Conference on Machine Learning and Machine Intelligence. Hangzhou, China: ACM, 2020. 24?30
[52]	鄭凱, 李建勝. 基于深度神經(jīng)網(wǎng)絡(luò )的圖像語(yǔ)義分割綜述. 測繪與空間地理信息, 2020, 43(10): 119?125 doi: 10.3969/j.issn.1672-5867.2020.10.032 Zheng Kai, Li Jian-Sheng. An overview of image semantic segmentation based on deep learning. Geomatics & Spatial Information Technology, 2020, 43(10): 119?125 doi: 10.3969/j.issn.1672-5867.2020.10.032
[53]	Belongie S, Malik J, Puzicha J. Shape context: A new descriptor for shape matching and object recognition. In: Proceedings of the 13th International Conference on Neural Information Processing Systems. Denver, USA: MIT Press, 2000. 798?804
[54]	Qi C R, Yi L, Su H, Guibas L J. PointNet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 652?660
[55]	Qi C R, Yi L, Su H, Guibas L J. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: Curran Associates Inc., 2017. 5105?5114
[56]	Ma X, Qin C, You H X, Ran H X, Fu Y. Rethinking network design and local geometry in point cloud: A simple residual MLP framework. In: Proceedings of the 10th International Conference on Learning Representations (ICLR). OpenReview.net, 2022.
[57]	Qian G C, Li Y C, Peng H W, Mai J J, Al Kader Hammoud H A, Elhoseiny M, et al. PointNeXt: Revisiting PointNet++ with improved training and scaling strategies. In: Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS). 2022. 23192?23204
[58]	Jain P, Tyagi V. LAPB: Locally adaptive patch-based wavelet domain edge-preserving image denoising. Information Sciences, 2015, 294: 164?181 doi: 10.1016/j.ins.2014.09.060
[59]	Li A B, Ma H W, Xu S H. Three-dimensional morphology and watershed-algorithm-based method for pitting corrosion evaluation. Buildings, 2022, 12(11): Article No. 1908 doi: 10.3390/buildings12111908
[60]	郭皓然, 邵偉, 周阿維, 楊宇祥, 劉凱斌. 全局閾值自適應的高亮金屬表面缺陷識別新方法. 儀器儀表學(xué)報, 2017, 38(11): 2797?2804 Guo Hao-Ran, Shao Wei, Zhou A-Wei, Yang Yu-Xiang, Liu Kai-Bin. Novel defect recognition method based on adaptive global threshold for highlight metal surface. Chinese Journal of Scientific Instrument, 2017, 38(11): 2797?2804
[61]	魏愛(ài)東. 基于脈沖渦流熱成像的鋼板缺陷檢測研究. 電子測試, 2020, 34(7): 56?59 doi: 10.3969/j.issn.1000-8519.2020.07.020 Wei Ai-Dong. Thermal image defect extraction based on two image segmentation algorithms. Electronic Test, 2020, 34(7): 56?59 doi: 10.3969/j.issn.1000-8519.2020.07.020
[62]	Prabha P A, Bharathwaj M, Dinesh K, Prashath G H. Defect detection of industrial products using image segmentation and saliency. Journal of Physics: Conference Series, 2021, 1916: Article No. 012165
[63]	Zhang X W, Ding Y Q, Lv Y Y, Shi A Y, Liang R Y. A vision inspection system for the surface defects of strongly reflected metal based on multi-class SVM. Expert Systems With Applications, 2011, 38(5): 5930?5939 doi: 10.1016/j.eswa.2010.11.030
[64]	Ghosh D, Kaabouch N. A survey on image mosaicing techniques. Journal of Visual Communication and Image Representation, 2016, 34: 1?11 doi: 10.1016/j.jvcir.2015.10.014
[65]	Lu R S, Shi Y Q, Li Q, Yu Q P. AOI techniques for surface defect inspection. In: Proceedings of the Applied Mechanics and Materials. Trans Tech Publications Ltd, 2010. 297?302
[66]	Kong Q, Wu Z, Song Y. Online detection of external thread surface defects based on an improved template matching algorithm. Measurement, 2022, 195: Article No. 111087
[67]	Pang G, Shen C, Cao L, Hengel V A D. Deep learning for anomaly detection: A review. ACM Computing Surveys, 2021, 54(2): 1?38
[68]	Konovalenko I, Maruschak P, Brevus V, Prentkovskis O. Recognition of scratches and abrasions on metal surfaces using a classifier based on a convolutional neural network. Metals, 2021, 11(4): Article No. 549 doi: 10.3390/met11040549
[69]	Jiang Q S, Tan D P, Li Y B, Ji S M, Cai C P, Zheng Q M. Object detection and classification of metal polishing shaft surface defects based on convolutional neural network deep learning. Applied Sciences, 2019, 10(1): Article No. 87 doi: 10.3390/app10010087
[70]	Tabernik D, ?ela S, Skvar? J, Sko?aj D. Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing, 2019, 31(3): 759?776
[71]	楊潔. 基于深度學(xué)習的無(wú)監督圖像異常模式檢測與識別研究 [博士學(xué)位論文], 中國科學(xué)院大學(xué), 中國, 2021. Yang Jie. Unsupervised Visual Anomaly Detection and Recognition Based on Deep Learning [Ph.D. dissertation], University of Chinese Academy of Sciences, China, 2021.
[72]	Chetverikov D, Khenokh Y. Matching for shape defect detection. In: Proceedings of the 8th International Conference on Computer Analysis of Images and Patterns (CAIP). Ljubljana, Slovenia: Springer, 1999. 367?374
[73]	Wang L, Pavlidis T. Direct gray-scale extraction of features for character recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15(10): 1053?1067 doi: 10.1109/34.254062
[74]	Chui H, Rangarajan A. A new point matching algorithm for non-rigid registration. Computer Vision and Image Understanding, 2003, 89(2): 114?141 doi: 10.1016/S1077-3142(03)00009-2
[75]	Borgefors G. Hierarchical chamfer matching: A parametric edge matching algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1988, 10(6): 849?865 doi: 10.1109/34.9107
[76]	Zhang H J, Hu Q. Fast image matching based-on improved SURF algorithm. In: Proceedings of the International Conference on Electronics, Communications and Control (ICECC). Ningbo, China: IEEE, 2011. 1460?1463
[77]	Crispin A J, Rankov V. Automated inspection of PCB components using a genetic algorithm template-matching approach. The International Journal of Advanced Manufacturing Technology, 2007, 35(3): 293?300
[78]	Hashemi N S, Aghdam R B, Ghiasi A S B, Fatemi P. Template matching advances and applications in image analysis. arXiv preprint arXiv: 1610.07231, 2016.
[79]	Wang H Y, Zhang J W, Tian Y, Chen H Y, Sun H X, Liu K. A simple guidance template-based defect detection method for strip steel surfaces. IEEE Transactions on Industrial Informatics, 2019, 15(5): 2798?2809 doi: 10.1109/TII.2018.2887145
[80]	Pernkopf F. Detection of surface defects on raw steel blocks using Bayesian network classifiers. Pattern Analysis and Applications, 2004, 7(3): 333?342 doi: 10.1007/s10044-004-0232-3
[81]	Aghdam S R, Amid E, Imani M F. A fast method of steel surface defect detection using decision trees applied to LBP based features. In: Proceedings of the 7th IEEE Conference on Industrial Electronics and Applications (ICIEA). Singapore: IEEE, 2012. 1447?1452
[82]	Samy M P, Foong S, Soh G S, Yeo K S. Automatic optical & laser-based defect detection and classification in brick masonry walls. In: Proceedings of the IEEE Region 10 Conference (TENCON). Singapore: IEEE, 2016. 3521?3524
[83]	Li X G, Zhu J, Shi H R, Cong Z J. Surface defect detection of seals based on K-means clustering algorithm and particle swarm optimization. Scientific Programming, 2021, 2021: Article No. 3965247
[84]	Yue B, Wang Y P, Min Y Z, Zhang Z H, Wang W R, Yong J. Rail surface defect recognition method based on AdaBoost multi-classifier combination. In: Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). Lanzhou, China: IEEE, 2019. 391?396
[85]	Soukup D, Huber-M?rk R. Convolutional neural networks for steel surface defect detection from photometric stereo images. In: Proceedings of the 10th International Symposium on Visual Computing (ISVC). Las Vegas, USA: Springer, 2014. 668?677
[86]	Masci J, Meier U, Ciresan D, Schmidhuber J, Fricout G. Steel defect classification with max-pooling convolutional neural networks. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN). Brisbane, Australia: IEEE, 2012. 1?6
[87]	Staar B, Lütjen M, Freitag M. Anomaly detection with convolutional neural networks for industrial surface inspection. Procedia CIRP, 2019, 79: 484?489 doi: 10.1016/j.procir.2019.02.123
[88]	Liao S C, Shao L. Graph sampling based deep metric learning for generalizable person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE, 2022. 7349?7358
[89]	Alzu'bi A, Albalas F, Al-Hadhrami T, Younis L B, Bashayreh A. Masked face recognition using deep learning: A review. Electronics, 2021, 10(21): Article No. 2666 doi: 10.3390/electronics10212666
[90]	Mordia R, Verma A K. Visual techniques for defects detection in steel products: A comparative study. Engineering Failure Analysis, 2022, 134: Article No. 106047 doi: 10.1016/j.engfailanal.2022.106047
[91]	Kim M S, Park T, Park P. Classification of steel surface defect using convolutional neural network with few images. In: Proceedings of the 12th Asian Control Conference (ASCC). Kitakyushu, Japan: IEEE, 2019. 1398?1401
[92]	Wu S L, Wu Y B, Cao D H, Zheng C Y. A fast button surface defect detection method based on Siamese network with imbalanced samples. Multimedia Tools and Applications, 2019, 78(24): 34627?34648 doi: 10.1007/s11042-019-08042-w
[93]	Li X, Yang X, Ma Z, Xue J H. Deep metric learning for few-shot image classification: A selective review. arXiv preprint arXiv: 2105.08149, 2021.
[94]	Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014. 580?587
[95]	He K M, Zhang X Y, Ren S Q, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904?1916 doi: 10.1109/TPAMI.2015.2389824
[96]	Girshick R. Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015. 1440?1448
[97]	Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137?1149 doi: 10.1109/TPAMI.2016.2577031
[98]	He K M, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017. 2980?2988
[99]	Cai Z W, Vasconcelos N. Cascade R-CNN: Delving into high quality object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 6154?6162
[100]	Guo F, Qian Y, Rizos D, Suo Z, Chen X B. Automatic rail surface defects inspection based on mask R-CNN. Transportation Research Record: Journal of the Transportation Research Board, 2021, 2675(11): 655?668 doi: 10.1177/03611981211019034
[101]	Xu Y Y, Li D W, Xie Q, Wu Q Y, Wang J. Automatic defect detection and segmentation of tunnel surface using modified mask R-CNN. Measurement, 2021, 178: Article No. 109316 doi: 10.1016/j.measurement.2021.109316
[102]	Fang J T, Tan X Y, Wang Y H. ACRM: Attention cascade R-CNN with Mix-NMS for metallic surface defect detection. In: Proceedings of the 25th International Conference on Pattern Recognition (ICPR). Milan, Italy: IEEE, 2021. 423?430
[103]	Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016. 779?788
[104]	Bochkovskiy A, Wang C Y, Liao H Y M. YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv: 2004.10934, 2020.
[105]	Li M J, Wang H, Wan Z B. Surface defect detection of steel strips based on improved YOLOv4. Computers and Electrical Engineering, 2022, 102: Article No. 108208 doi: 10.1016/j.compeleceng.2022.108208
[106]	Usamentiaga R, Lema D G, Pedrayes O D, Garcia D F. Automated surface defect detection in metals: A comparative review of object detection and semantic segmentation using deep learning. IEEE Transactions on Industry Applications, 2022, 58(3): 4203?4213 doi: 10.1109/TIA.2022.3151560
[107]	Xu Y H, Wang X J, Li S Y. Track surface defect detection based on EfficientDet. In: Proceedings of the 5th International Conference on Electrical Engineering and Information Technologies for Rail Transportation (EITRT) 2021: Rail Transportation Information Processing and Operational Management Technologies. Singapore: Springer, 2022. 56?66
[108]	Wang C Y, Bochkovskiy A, Liao H Y M. Scaled-YOLOv4: Scaling cross stage partial network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE, 2021. 13024?13033
[109]	He Y, Song K C, Meng Q G, Yan Y H. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Transactions on Instrumentation and Measurement, 2020, 69(4): 1493?1504 doi: 10.1109/TIM.2019.2915404
[110]	Anthony A, Ho E S L, Woo W L, Gao B. A review and benchmark on state-of-the-art steel defects detection [Online], available: http://dx.doi.org/10.2139/ssrn.4121951, March 19, 2023
[111]	Song G L, Liu Y, Wang X G. Revisiting the sibling head in object detector. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 11560?11569
[112]	Lin T Y, Goyal P, Girshick R, He K M, Dollár P. Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017. 2999?3007
[113]	Ma P F, Ma J, Wang X J, Yang L C, Wang N N. Deformable convolutional networks for multi-view 3D shape classification. Electronics Letters, 2018, 54(24): 1373?1375 doi: 10.1049/el.2018.6851
[114]	Law H, Deng J. CornerNet: Detecting objects as paired keypoints. In: Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018. 765?781
[115]	Ge Z, Liu S T, Wang F, Li Z M, Sun J. YOLOX: Exceeding YOLO series in 2021. arXiv preprint arXiv: 2107.08430, 2021.
[116]	Zhu B J, Wang J F, Jiang Z K, Zong F H, Liu S T, Li Z M, et al. AutoAssign: Differentiable label assignment for dense object detection. arXiv preprint arXiv: 2007.03496, 2020.
[117]	Lu H T, Fang M Y, Qiu Y, Xu W Q. An anchor-free defect detector for complex background based on pixelwise adaptive multiscale feature fusion. IEEE Transactions on Instrumentation and Measurement, 2023, 72: Article No. 5002312
[118]	Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S. End-to-end object detection with transformers. In: Proceedings of the 16th European Conference on Computer Vision (ECCV). Glasgow, UK: Springer, 2020. 213?229
[119]	Zhu X, Su W, Lu L, Li B, Wang X, Dai J. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv: 2010.04159, 2020.
[120]	Misra I, Girdhar R, Joulin A. An end-to-end transformer model for 3D object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, Canada: IEEE, 2021. 2886?2897
[121]	Lv X, Duan F, Jiang J, Fu X, Gan L. Deep metallic surface defect detection: The new benchmark and detection network. Sensors, 2020, 20(6): Article No. 1562
[122]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE, 2015. 3431?3440
[123]	Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In: Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Munich, Germany: Springer, 2015. 234?241
[124]	Sabet D N, Zarifi M R, Khoramdel J, Borhani Y, Najafi E. An automated visual defect segmentation for flat steel surface using deep neural networks. In: Proceedings of the 12th International Conference on Computer and Knowledge Engineering (ICCKE). Mashhad, Islamic Republic of Iran: IEEE, 2022. 423?427
[125]	Lin T Y, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 2117?2125
[126]	Xie S N, Girshick R, Dollár P, Tu Z W, He K M. Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 5987?5995
[127]	Koonce B. EfficientNet. Convolutional Neural Networks With Swift for Tensorflow: Image Recognition and Dataset Categorization. Berkeley: Apress, 2021. 109?123
[128]	Chen L C, Papandreou G, Kokkinos I, Murphy K, Yuille A L. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834?848 doi: 10.1109/TPAMI.2017.2699184
[129]	Zheng S, Jayasumana S, Romera-Paredes B, Vineet V, Su Z Z, Du D L, et al. Conditional random fields as recurrent neural networks. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015. 1529?1537
[130]	Ma L F, Li J. SD-GCN: Saliency-based dilated graph convolution network for pavement crack extraction from 3D point clouds. International Journal of Applied Earth Observation and Geoinformation, 2022, 111: Article No. 102836 doi: 10.1016/j.jag.2022.102836
[131]	Huang Y B, Qiu C Y, Guo Y, Wang X N, Yuan K. Surface defect saliency of magnetic tile. In: Proceedings of the 14th International Conference on Automation Science and Engineering (CASE). Munich, Germany: IEEE, 2018. 612?617
[132]	Tian H, Li F. Autoencoder-based fabric defect detection with cross-patch similarity. In: Proceedings of the 16th International Conference on Machine Vision Applications (MVA). Tokyo, Japan: IEEE, 2019. 1?6
[133]	Mei S, Yang H, Yin Z P. An unsupervised-learning-based approach for automated defect inspection on textured surfaces. IEEE Transactions on Instrumentation and Measurement, 2018, 67(6): 1266?1277 doi: 10.1109/TIM.2018.2795178
[134]	Huang C Q, Cao J K, Ye F, Li M S, Zhang Y, Lu C W. Inverse-transform Autoencoder for anomaly detection. arXiv preprint arXiv: 1911.10676, 2019.
[135]	Zimmerer D, Petersen J, Kohl S A A, Maier-Hein K H. A case for the score: Identifying image anomalies using variational autoencoder gradients. arXiv preprint arXiv: 1912.00003, 2019.
[136]	Kwon G, Prabhushankar M, Temel D, AlRegib G. Backpropagated gradient representations for anomaly detection. In: Proceedings of the 16th European Conference on Computer Vision (ECCV). Glasgow, UK: Springer, 2020. 206?226
[137]	Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative adversarial networks. Communications of the ACM, 2020, 63(11): 139?144 doi: 10.1145/3422622
[138]	Baur C, Denner S, Wiestler B, Navab N, Albarqouni S. Autoencoders for unsupervised anomaly segmentation in brain MR images: A comparative study. Medical Image Analysis, 2021, 69: Article No. 101952 doi: 10.1016/j.media.2020.101952
[139]	Schlegl T, Seeb?ck P, Waldstein S M, Langs G, Schmidt-Erfurth U. f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks. Medical Image Analysis, 2019, 54: 30?44 doi: 10.1016/j.media.2019.01.010
[140]	Rudolph M, Wehrbein T, Rosenhahn B, Wandt B. Fully convolutional cross-scale-flows for image-based defect detection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa, USA: IEEE, 2022. 1829?1838
[141]	Yu J W, Zheng Y, Wang X, Li W, Wu Y S, Zhao R, et al. FastFlow: Unsupervised anomaly detection and localization via 2D normalizing flows. arXiv preprint arXiv: 2111.07677, 2021.
[142]	Yi J H, Yoon S. Patch SVDD: Patch-level SVDD for anomaly detection and segmentation. In: Proceedings of the 15th Asian Conference on Computer Vision (ACCV). Kyoto, Japan: Springer, 2021. 375?390
[143]	Tax D M J, Duin R P W. Support vector data description. Machine Learning, 2004, 54(1): 45?66 doi: 10.1023/B:MACH.0000008084.60811.49
[144]	Yang M H, Wu P, Feng H. MemSeg: A semi-supervised method for image surface defect detection using differences and commonalities. Engineering Applications of Artificial Intelligence, 2023, 119: Article No. 105835 doi: 10.1016/j.engappai.2023.105835
[145]	Roth K, Pemula L, Zepeda J, Sch?lkopf B, Brox T, Gehler P. Towards total recall in industrial anomaly detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE, 2022. 14298?14308
[146]	Bae J, Lee J H, Kim S. Image anomaly detection and localization with position and neighborhood information. arXiv preprint arXiv: 2211.12634, 2022.
[147]	Bergmann P, Fauser M, Sattlegger D, Steger C. Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 4182?4191
[148]	Batzner K, Heckler L, K?nig R. EfficientAD: Accurate visual anomaly detection at millisecond-level latencies. arXiv preprint arXiv: 2303.14535, 2023.