基于序列注意力和局部相位引導的骨超聲圖像分割網(wǎng)絡(luò )

陳芳; 張道強; 廖洪恩; 趙喆

doi:10.16383/j.aas.c210298

基于序列注意力和局部相位引導的骨超聲圖像分割網(wǎng)絡(luò )

doi: 10.16383/j.aas.c210298

陳芳^1,,
張道強^1,,
廖洪恩^2,,
趙喆^3,

1.
南京航空航天大學(xué)計算機科學(xué)與技術(shù)學(xué)院南京 211106
2.
清華大學(xué)醫學(xué)院北京 100084
3.
清華大學(xué)附屬北京清華長(cháng)庚醫院骨科與運動(dòng)醫學(xué)中心北京 102218

基金項目: 國家自然科學(xué)基金(U20A20389, 61901214), 中國博士后科學(xué)基金(2021T140322, 2020M671484)資助

詳細信息

作者簡(jiǎn)介:
陳芳：南京航空航天大學(xué)計算機科學(xué)與技術(shù)學(xué)院副教授. 主要研究方向為醫學(xué)圖像處理, 計算機輔助手術(shù)導航. 本文通信作者. E-mail: chenfang@nuaa.edu.cn

張道強：南京航空航天大學(xué)計算機科學(xué)與技術(shù)學(xué)院教授. 主要研究方向為醫學(xué)圖像處理, 機器學(xué)習. E-mail: dqzhang@nuaa.edu.cn

廖洪恩：清華大學(xué)醫學(xué)院教授. 主要研究方向為三維成像, 微創(chuàng )診療. E-mail: liao@tsinghua.edu.cn

趙喆：清華大學(xué)附屬北京清華長(cháng)庚醫院骨科與運動(dòng)醫學(xué)中心副主任醫師, 清華大學(xué)臨床醫學(xué)院副教授. 主要研究方向為創(chuàng )傷骨科, 計算機導航手術(shù), 骨科植入物設計. E-mail: zhaozhao_02@163.com

計量
- 文章訪(fǎng)問(wèn)數: 705
- HTML全文瀏覽量: 525
- PDF下載量: 155
- 被引次數: 0
出版歷程
- 收稿日期: 2021-04-08
- 錄用日期: 2021-11-26
- 網(wǎng)絡(luò )出版日期: 2022-02-05
- 刊出日期: 2024-05-29

Bone Ultrasound Segmentation Network Based on Sequential Attention and Local Phase Guidance

1.
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106
2.
School of Medicine, Tsinghua University, Beijing 100084
3.
Orthopaedics and Sports Medicine Center, Tsinghua University Affiliated Beijing Tsinghua Changgung Hospital, Beijing 102218

Funds: Supported by National Natural Science Foundation of China (U20A20389, 61901214) and China Postdoctoral Science Foundation (2021T140322, 2020M671484)

More Information

Author Bio:
CHEN Fang　Associate professor at the College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics. Her research interest covers medical image processing and image-guided surgery. Corresponding author of this paper

ZHANG Dao-Qiang　Professor at the College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics. His research interest covers medical image processing and machine learning

LIAO Hong-En　Professor at the School of Medicine, Tsinghua University. His research interest covers 3D image and minimally invasive diagnosis and therapy

ZHAO Zhe　Associate chief physician of the Orthopaedics and Sports Medicine Center, Tsinghua University Affiliated Beijing Tsinghua Changgung Hospital and associate professor at the School of Clinical Medicine, Tsinghua University. His research interest covers orthopaedic trauma, computer assisted surgery, and orthopaedics implant development

摘要

摘要: 在超聲輔助的骨科手術(shù)導航中, 需要從采集的超聲圖像序列中精確分割出骨結構, 并展示給醫生, 來(lái)輔助醫生進(jìn)行術(shù)中決策. 但是, 圖像噪聲、成像偽影以及模糊的骨邊界導致從超聲圖像序列中精確分割提取骨結構十分困難. 為解決該問(wèn)題, 提出一種新的基于序列注意力與局部相位引導的骨超聲圖像分割網(wǎng)絡(luò ). 該網(wǎng)絡(luò )一方面自適應地利用超聲序列幀之間的關(guān)系即序列注意力來(lái)輔助骨結構的語(yǔ)義分割. 另一方面, 該網(wǎng)絡(luò )通過(guò)引入局部相位引導模塊, 突出骨邊緣信息, 進(jìn)一步提高分割精度. 利用包含19 050幅圖像的骨超聲數據集, 進(jìn)行交叉實(shí)驗、消融實(shí)驗并與最新的超聲骨分割方法進(jìn)行比較. 實(shí)驗結果表明所提方法對骨結構分割精度高, 優(yōu)于現有的超聲骨分割方法.
- 骨科導航 /
- 超聲圖像分割 /
- 局部相位 /
- 序列注意力
Abstract: In the ultrasound assisted navigation of orthopaedics, the bone structure needs to be segmented accurately from the collected ultrasound images and displayed to the doctor to assist the intraoperative decision-making. However, it is difficult to segment bone structures from ultrasound images because of imaging noises, shadow artifacts and blurred bone boundaries. For solving this problem, this paper proposes a bone ultrasound image segmentation network based on sequential attention and local phase guidance. On the one hand, the network adaptively uses the relationship between frames of ultrasound sequence, that is, sequence attention, to assist the semantic segmentation of bone structures. On the other hand, the local phase guidance module is introduced to highlight the bone edge information and further improve the segmentation accuracy. We performed the cross validation, ablation experiments and the comparison experiments with the state-of-arts by using a dataset that contained 19 050 bone ultrasound images. The experimental results show that the proposed method has high accuracy and is superior to the existing bone segmentation methods.
- Orthopaedics navigation /
- ultrasound image segmentation /
- local phase /
- sequence attention

HTML全文

圖 1 基于序列注意力與局部相位引導的骨超聲圖像分割網(wǎng)絡(luò )系統框圖; 圖中ConvA表示卷積核為1×1 、步長(cháng)為1的卷積操作; ConvB表示卷積核為3×3、步長(cháng)為1的卷積操作

Fig. 1 Bone ultrasound segmentation network based on sequential attention and local phase guidance; ConvA denotes the convolution operation with kernel size of 1×1 and a stride of 1; ConvB denotes the convolution operation with kernel size of 3×3 and a stride of 1

下載: 全尺寸圖片幻燈片

圖 2 骨超聲圖像序列采集示意圖

Fig. 2 Diagram of bone ultrasound sequence acquisition

下載: 全尺寸圖片幻燈片

圖 3 10折交叉實(shí)驗中訓練和測試超聲圖像幀數的分布

Fig. 3 The distributions of training and testing frames in 10-fold validation experiments

下載: 全尺寸圖片幻燈片

圖 4 10次交叉實(shí)驗的交并比IoU值的箱線(xiàn)圖

Fig. 4 Boxplots for IoU values in 10-fold validation experiments

下載: 全尺寸圖片幻燈片

圖 5 利用所提出的分割網(wǎng)絡(luò )對骨結構進(jìn)行分割的實(shí)際結果(第1行: 待分割的骨超聲圖像;第2行: 專(zhuān)家手動(dòng)標注的骨結構; 第3行: 利用所提出的方法自動(dòng)分割的骨結構)

Fig. 5 Bone segmentation results by using the proposed segmentation network (The first line: Ultrasound bone images to be segmented; The second line: Bone structures manually delineated by experts; The third line: Segmented bone structures using the proposed method)

下載: 全尺寸圖片幻燈片

圖 6 消融實(shí)驗結果(本文所提出的分割網(wǎng)絡(luò )與兩個(gè)變體模型的骨分割實(shí)驗結果比較)

Fig. 6 Ablation results (Comparison between our proposed network and two variants)

下載: 全尺寸圖片幻燈片

圖 7 局部相位模塊消融實(shí)驗結果示例. 第1幀: 超聲圖像幀; 第2幀: 手動(dòng)標注的骨結構;第3幀: 帶有局部相位模塊的模型分割結果; 第4幀: 去除局部相位模塊的模型分割結果

Fig. 7 Ablation results of local phase guidance (The first graph: Ultrasound image frame; The second graph: Manually delineated bone structures; The third graph: Results of the model with local phase guidance; The fourth graph: Results of the model without local phase guidance)

下載: 全尺寸圖片幻燈片

表 1 本研究采集的超聲圖像序列數據集的信息

Table 1 Information of the ultrasound image sequence dataset collected in this study

志愿者ID	圖像幀數量	圖像分辨率(mm/像素)
1	1 900	0.19 ~ 0.21
2	2 010	0.17 ~ 0.21
3	1 740	0.19 ~ 0.21
4	1 870	0.21 ~ 0.23
5	2 010	0.19 ~ 0.23
6	1 980	0.17 ~ 0.21
7	1 560	0.18 ~ 0.23
8	1 980	0.19 ~ 0.21
9	1 900	0.21 ~ 0.23
10	2 100	0.17 ~ 0.23

下載: 導出CSV

表 2 所提出的分割網(wǎng)絡(luò )對10名志愿者采集的超聲序列圖像的分割結果

Table 2 Results of our proposed model obtained on the ultrasound images from ten volunteers

Exp_K	交并比 IoU	平均歐氏距離 AED
Exp_1	0.91 ± 0.08	0.41 ± 0.06
Exp_2	0.90 ± 0.08	0.39 ± 0.07
Exp_3	0.91 ± 0.07	0.39 ± 0.06
Exp_4	0.92 ± 0.07	0.37 ± 0.05
Exp_5	0.90 ± 0.07	0.41 ± 0.08
Exp_6	0.89 ± 0.08	0.43 ± 0.08
Exp_7	0.91 ± 0.06	0.42 ± 0.07
Exp_8	0.90 ± 0.07	0.41 ± 0.09
Exp_9	0.92 ± 0.06	0.40 ± 0.07
Exp_10	0.91 ± 0.08	0.39 ± 0.09

下載: 導出CSV

表 3 針對主干網(wǎng)絡(luò )的比較實(shí)驗結果

Table 3 Comparison experiments by using different backbones

主干網(wǎng)絡(luò )	ResNet18	ResNet34	ResNet50	ResNet101	VGGNet
超聲圖像骨分割平均交并比IoU值	0.89 ± 0.10	0.90 ± 0.08	0.91 ± 0.07	0.89 ± 0.09	0.89 ± 0.09

下載: 導出CSV

表 4 本文所提出的超聲圖像分割網(wǎng)絡(luò )與其他最新的分割方法的實(shí)驗結果(IoU值)比較

Table 4 Comparison of the experimental results (IoU values) of the ultrasound image segmentation network proposed in this study with other state-of-the-art segmentation methods

實(shí)驗結果/ 方法	局部相位引導 CNN	BoneNet模型	濾波層引導 CNN	時(shí)空CNN	注意引導網(wǎng)絡(luò ) AGNet	三重注意力網(wǎng)絡(luò ) TriANet	本文方法
Exp_1	0.88	0.88	0.87	0.85	0.85	0.86	0.91
Exp_2	0.87	0.88	0.86	0.84	0.85	0.85	0.90
Exp_3	0.87	0.89	0.87	0.86	0.86	0.88	0.91
Exp_4	0.89	0.89	0.87	0.85	0.86	0.87	0.92
Exp_5	0.87	0.89	0.85	0.86	0.87	0.87	0.90
Exp_6	0.86	0.88	0.85	0.84	0.85	0.85	0.89
Exp_7	0.86	0.90	0.86	0.84	0.86	0.87	0.91
Exp_8	0.87	0.88	0.84	0.85	0.86	0.86	0.90
Exp_9	0.89	0.91	0.89	0.86	0.87	0.88	0.92
Exp_10	0.87	0.89	0.87	0.87	0.87	0.89	0.91
平均值	0.87 ± 0.13	0.88 ± 0.08	0.86 ± 0.14	0.87 ± 0.12	0.86 ± 0.09	0.88 ± 0.10	0.91 ± 0.07

下載: 導出CSV

<button id="qm3rj"><thead id="qm3rj"></thead></button>

亚洲第一网址_国产国产人精品视频69_久久久久精品视频_国产精品第九页

參考文獻(30)

[1]	Zhe Z, Zhu J J, Song F, He D W, Deng J Z, Chen F, et al. Intraoperative ultrasound-guided reduction of femoral shaft fractures using intramedullary nailing: A technical note. Archives of Orthopaedic and Trauma Surgery, 2019, 139(5): 589?596 doi: 10.1007/s00402-018-3085-8
[2]	Zhou H, Zhang G, Li M, Qu X Y, Cao Y J, Liu X, et al. Ultrasonography-guided closed reduction in the treatment of displaced transphyseal fracture of the distal humerus. Journal of Orthopaedic Surgery and Research, 2020, 15(1): Article No. 575 doi: 10.1186/s13018-020-02118-2
[3]	Wein W, Karamalis A, Baumgartner A, Navab N. Automatic bone detection and soft tissue aware ultrasound-CT registration for computer-aided orthopedic surgery. International Journal of Computer Assisted Radiology and Surgery, 2015, 10(6): 971?979 doi: 10.1007/s11548-015-1208-z
[4]	Hacihaliloglu I. Ultrasound imaging and segmentation of bone surfaces: A review. Technology, 2017, 5(2): 74?80 doi: 10.1142/S2339547817300049
[5]	Pandey P U, Quader N, Guy P, Garbi R, Hodgson A J. Ultrasound bone segmentation: A scoping review of techniques and validation practices. Ultrasound in Medicine and Biology, 2020, 46(4): 921?935 doi: 10.1016/j.ultrasmedbio.2019.12.014
[6]	Masson-Sibut A, Nakib A, Petit E, Leitner F. Computer-assisted intramedullary nailing using real-time bone detection in 2D ultrasound images. In: Proceedings of the 2nd International Workshop on Machine Learning in Medical Imaging. Toronto, Canada: Springer, 2011. 18?25
[7]	Wang P Y, Patel V M, Hacihaliloglu I. Simultaneous segmentation and classification of bone surfaces from ultrasound using a multi-feature guided CNN. In: Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada, Spain: Springer, 2018. 134?142
[8]	Huang Z X, Wang L W, Leung F H F, Banerjee S, Yang D, Lee T, et al. Bone feature segmentation in ultrasound spine image with robustness to speckle and regular occlusion noise. In: Proceedings of IEEE International Conference on Systems, Man, and Cybernetics (SMC). Toronto, Canada: IEEE, 2020. 1566?1571
[9]	Hacihaliloglu I, Abugharbieh R, Hodgson A, Rohling R. Bone segmentation and fracture detection in ultrasound using 3D local phase features. In: Proceedings of the 11th International Conference on Medical Image Computing and Computer-Assisted Intervention. New York, USA: Springer, 2008. 287?295
[10]	范家偉, 張如如, 陸萌, 何佳雯, 康霄陽(yáng), 柴文俊, 等. 深度學(xué)習方法在糖尿病視網(wǎng)膜病變診斷中的應用. 自動(dòng)化學(xué)報, 2021, 47(5): 985?1004 Fan Jia-Wei, Zhang Ru-Ru, Lu Meng, He Jia-Wen, Kang Xiao-Yang, Chai Wen-Jun, et al. Applications of deep learning techniques for diabetic retinal diagnosis. Acta Automatica Sinica, 2021, 47(5): 985?1004
[11]	蔣蕓, 譚寧. 基于條件深度卷積生成對抗網(wǎng)絡(luò )的視網(wǎng)膜血管分割. 自動(dòng)化學(xué)報, 2021, 47(1): 136?147 Jiang Yun, Tan Ning. Retinal vessel segmentation based on conditional deep convolutional generative adversarial networks. Acta Automatica Sinica, 2021, 47(1): 136?147
[12]	夏平, 施宇, 雷幫軍, 龔國強, 胡蓉, 師冬霞. 復小波域混合概率圖模型的超聲醫學(xué)圖像分割. 自動(dòng)化學(xué)報, 2021, 47(1): 185?196 Xia Ping, Shi Yu, Lei Bang-Jun, Gong Guo-Qiang, Hu Rong, Shi Dong-Xia. Ultrasound medical image segmentation based on hybrid probabilistic graphical model in complex-wavelet domain. Acta Automatica Sinica, 2021, 47(1): 185?196
[13]	Ouahabi A, Taleb-Ahmed A. RETRACTED: Deep learning for real-time semantic segmentation: Application in ultrasound imaging. Pattern Recognition Letters, 2021, 144: 27?34 doi: 10.1016/j.patrec.2021.01.010
[14]	Baka N, Leenstra S, van Walsum T. Ultrasound aided vertebral level localization for lumbar surgery. IEEE Transactions on Medical Imaging, 2017, 36(10): 2138?2147 doi: 10.1109/TMI.2017.2738612
[15]	Ciganovic M, ?zdemir F, Farshad M, G?ksel O. Deep learning techniques for bone surface delineation in ultrasound. In: Proceedings of the SPIE 10955, Medical Imaging 2019: Ultrasonic Imaging and Tomography. San Diego, USA: SPIE, 2019. Article No. 109550Y
[16]	Wang P Y, Vives M, Patel V M, Hacihaliloglu I. Robust real-time bone surfaces segmentation from ultrasound using a local phase tensor-guided CNN. International Journal of Computer Assisted Radiology and Surgery, 2020, 15(7): 1127?1135 doi: 10.1007/s11548-020-02184-1
[17]	Alsinan A Z, Patel V M, Hacihaliloglu I. Automatic segmentation of bone surfaces from ultrasound using a filter-layer-guided CNN. International Journal of Computer Assisted Radiology and Surgery, 2019, 14(5): 775?783 doi: 10.1007/s11548-019-01934-0
[18]	Luan K, Li Z Y, Li J. An efficient end-to-end CNN for segmentation of bone surfaces from ultrasound. Computerized Medical Imaging and Graphics, 2020, 84: Article No. 101766 doi: 10.1016/j.compmedimag.2020.101766
[19]	Hu P, Caba F, Wang O, Lin Z, Sclaroff S, Perazzi F. Temporally distributed networks for fast video semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020. 8815?8824
[20]	Yao R, Xu X, Zhou Y, Zhao J Q, Fang L. Joint attention mechanism for unsupervised video object segmentation. In: Proceedings of the 4th Chinese Conference on Pattern Recognition and Computer Vision (PRCV). Beijing, China: Springer, 2021. 154?165
[21]	Alcázar J L, Bravo M A, Jeanneret G, Thabet A K, Brox T, Arbeláez P, et al. MAIN: Multi-attention instance network for video segmentation. Computer Vision and Image Understanding, 2021, 210: Article No. 103240 doi: 10.1016/j.cviu.2021.103240
[22]	He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 770?778
[23]	He K M, Zhang X Y, Ren S Q, Sun J. Identity mappings in deep residual networks. In: Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer, 2016. 630?645
[24]	Cox J, Rubin S, Adams J, Pereira C, Dighe M, Alessio A. Hyperparameter selection for ResNet classification of malignancy from thyroid ultrasound images. In: Proceedings of the SPIE 11314, Medical Imaging 2020: Computer-Aided Diagnosis. Houston, USA: SPIE, 2020. Article No. 1131447
[25]	Zhang Q, Cui Z P, Niu X G, Geng S J, Qiao Y. Image segmentation with pyramid dilated convolution based on ResNet and U-Net. In: Proceedings of the 24th International Conference on Neural Information Processing. Guangzhou, China: Springer, 2017. 364?372
[26]	Hacihaliloglu I, Rasoulian A, Rohling R N, Abolmaesumi P. Local phase tensor features for 3-D ultrasound to statistical shape+pose spine model registration. IEEE Transactions on Medical Imaging, 2014, 33(11): 2167?2179 doi: 10.1109/TMI.2014.2332571
[27]	Hacihaliloglu I. Localization of bone surfaces from ultrasound data using local phase information and signal transmission maps. In: Proceedings of the 5th International Workshop on Computational Methods and Clinical Applications in Musculoskeletal Imaging. Quebec City, Canada: Springer, 2017. 1?11
[28]	Xu K, Wen L Y, Li G R, Bo L F, Huang Q M. Spatiotemporal CNN for video object segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019. 1379?1388
[29]	Li J Y, Zhao Y K, Fu J, Wu J J, Liu J. Attention-guided network for semantic video segmentation. IEEE Access, 2019, 7: 140680?140689 doi: 10.1109/ACCESS.2019.2943365
[30]	Tian Y, Zhang Y J, Zhou D, Cheng G H, Chen W G, Wang R L. Triple attention network for video segmentation. Neurocomputing, 2020, 417: 202?211 doi: 10.1016/j.neucom.2020.07.078