基于序列注意力和局部相位引導的骨超聲圖像分割網絡

陳芳; 張道強; 廖洪恩; 趙喆

doi:10.16383/j.aas.c210298

基于序列注意力和局部相位引導的骨超聲圖像分割網絡

doi: 10.16383/j.aas.c210298

陳芳^1,,
張道強^1,,
廖洪恩^2,,
趙喆^3,

1.
南京航空航天大學計算機科學與技術學院南京 211106
2.
清華大學醫學院北京 100084
3.
清華大學附屬北京清華長庚醫院骨科與運動醫學中心北京 102218

基金項目: 國家自然科學基金(U20A20389, 61901214), 中國博士后科學基金(2021T140322, 2020M671484)資助

詳細信息

作者簡介:
陳芳：南京航空航天大學計算機科學與技術學院副教授. 主要研究方向為醫學圖像處理, 計算機輔助手術導航. 本文通信作者. E-mail: chenfang@nuaa.edu.cn

張道強：南京航空航天大學計算機科學與技術學院教授. 主要研究方向為醫學圖像處理, 機器學習. E-mail: dqzhang@nuaa.edu.cn

廖洪恩：清華大學醫學院教授. 主要研究方向為三維成像, 微創診療. E-mail: liao@tsinghua.edu.cn

趙喆：清華大學附屬北京清華長庚醫院骨科與運動醫學中心副主任醫師, 清華大學臨床醫學院副教授. 主要研究方向為創傷骨科, 計算機導航手術, 骨科植入物設計. E-mail: zhaozhao_02@163.com

計量
- 文章訪問數: 593
- HTML全文瀏覽量: 442
- PDF下載量: 99
- 被引次數: 0
出版歷程
- 收稿日期: 2021-04-08
- 錄用日期: 2021-11-26
- 網絡出版日期: 2022-02-05
- 刊出日期: 2024-05-20

Bone Ultrasound Segmentation Network Based on Sequential Attention and Local Phase Guidance

1.
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106
2.
School of Medicine, Tsinghua University, Beijing 100084
3.
Orthopaedics and Sports Medicine Center, Tsinghua University Affiliated Beijing Tsinghua Changgung Hospital, Beijing 102218

Funds: Supported by National Natural Science Foundation of China (U20A20389, 61901214) and China Postdoctoral Science Foundation (2021T140322, 2020M671484)

More Information

Author Bio:
CHEN Fang　Associate professor at the College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics. Her research interest covers medical image processing and image-guided surgery. Corresponding author of this paper

ZHANG Dao-Qiang　Professor at the College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics. His research interest covers medical image processing and machine learning

LIAO Hong-En　Professor at the School of Medicine, Tsinghua University. His research interest covers 3D image and minimally invasive diagnosis and therapy

ZHAO Zhe　Associate chief physician of the Orthopaedics and Sports Medicine Center, Tsinghua University Affiliated Beijing Tsinghua Changgung Hospital and associate professor at the School of Clinical Medicine, Tsinghua University. His research interest covers orthopaedic trauma, computer assisted surgery, and orthopaedics implant development

摘要

摘要: 在超聲輔助的骨科手術導航中, 需要從采集的超聲圖像序列中精確分割出骨結構, 并展示給醫生, 來輔助醫生進行術中決策. 但是, 圖像噪聲、成像偽影以及模糊的骨邊界導致從超聲圖像序列中精確分割提取骨結構十分困難. 為解決該問題, 提出一種新的基于序列注意力與局部相位引導的骨超聲圖像分割網絡. 該網絡一方面自適應地利用超聲序列幀之間的關系即序列注意力來輔助骨結構的語義分割. 另一方面, 該網絡通過引入局部相位引導模塊, 突出骨邊緣信息, 進一步提高分割精度. 利用包含19 050幅圖像的骨超聲數據集, 進行交叉實驗、消融實驗并與最新的超聲骨分割方法進行比較. 實驗結果表明所提方法對骨結構分割精度高, 優于現有的超聲骨分割方法.
- 骨科導航 /
- 超聲圖像分割 /
- 局部相位 /
- 序列注意力
Abstract: In the ultrasound assisted navigation of orthopaedics, the bone structure needs to be segmented accurately from the collected ultrasound images and displayed to the doctor to assist the intraoperative decision-making. However, it is difficult to segment bone structures from ultrasound images because of imaging noises, shadow artifacts and blurred bone boundaries. For solving this problem, this paper proposes a bone ultrasound image segmentation network based on sequential attention and local phase guidance. On the one hand, the network adaptively uses the relationship between frames of ultrasound sequence, that is, sequence attention, to assist the semantic segmentation of bone structures. On the other hand, the local phase guidance module is introduced to highlight the bone edge information and further improve the segmentation accuracy. We performed the cross validation, ablation experiments and the comparison experiments with the state-of-arts by using a dataset that contained 19 050 bone ultrasound images. The experimental results show that the proposed method has high accuracy and is superior to the existing bone segmentation methods.
- Orthopaedics navigation /
- ultrasound image segmentation /
- local phase /
- sequence attention

HTML全文

圖 1 基于序列注意力與局部相位引導的骨超聲圖像分割網絡系統框圖; 圖中ConvA表示卷積核為1×1 、步長為1的卷積操作; ConvB表示卷積核為3×3、步長為1的卷積操作

Fig. 1 Bone ultrasound segmentation network based on sequential attention and local phase guidance; ConvA denotes the convolution operation with kernel size of 1×1 and a stride of 1; ConvB denotes the convolution operation with kernel size of 3×3 and a stride of 1

下載: 全尺寸圖片幻燈片

圖 2 骨超聲圖像序列采集示意圖

Fig. 2 Diagram of bone ultrasound sequence acquisition

下載: 全尺寸圖片幻燈片

圖 3 10折交叉實驗中訓練和測試超聲圖像幀數的分布

Fig. 3 The distributions of training and testing frames in 10-fold validation experiments

下載: 全尺寸圖片幻燈片

圖 4 10次交叉實驗的交并比IoU值的箱線圖

Fig. 4 Boxplots for IoU values in 10-fold validation experiments

下載: 全尺寸圖片幻燈片

圖 5 利用所提出的分割網絡對骨結構進行分割的實際結果(第1行: 待分割的骨超聲圖像;第2行: 專家手動標注的骨結構; 第3行: 利用所提出的方法自動分割的骨結構)

Fig. 5 Bone segmentation results by using the proposed segmentation network (The first line: Ultrasound bone images to be segmented; The second line: Bone structures manually delineated by experts; The third line: Segmented bone structures using the proposed method)

下載: 全尺寸圖片幻燈片

圖 6 消融實驗結果(本文所提出的分割網絡與兩個變體模型的骨分割實驗結果比較)

Fig. 6 Ablation results (Comparison between our proposed network and two variants)

下載: 全尺寸圖片幻燈片

圖 7 局部相位模塊消融實驗結果示例. 第1幀: 超聲圖像幀; 第2幀: 手動標注的骨結構;第3幀: 帶有局部相位模塊的模型分割結果; 第4幀: 去除局部相位模塊的模型分割結果

Fig. 7 Ablation results of local phase guidance (The first graph: Ultrasound image frame; The second graph: Manually delineated bone structures; The third graph: Results of the model with local phase guidance; The fourth graph: Results of the model without local phase guidance)

下載: 全尺寸圖片幻燈片

表 1 本研究采集的超聲圖像序列數據集的信息

Table 1 Information of the ultrasound image sequence dataset collected in this study

志愿者ID	圖像幀數量	圖像分辨率(mm/像素)
1	1 900	0.19 ~ 0.21
2	2 010	0.17 ~ 0.21
3	1 740	0.19 ~ 0.21
4	1 870	0.21 ~ 0.23
5	2 010	0.19 ~ 0.23
6	1 980	0.17 ~ 0.21
7	1 560	0.18 ~ 0.23
8	1 980	0.19 ~ 0.21
9	1 900	0.21 ~ 0.23
10	2 100	0.17 ~ 0.23

下載: 導出CSV

表 2 所提出的分割網絡對10名志愿者采集的超聲序列圖像的分割結果

Table 2 Results of our proposed model obtained on the ultrasound images from ten volunteers

Exp_K	交并比 IoU	平均歐氏距離 AED
Exp_1	0.91 ± 0.08	0.41 ± 0.06
Exp_2	0.90 ± 0.08	0.39 ± 0.07
Exp_3	0.91 ± 0.07	0.39 ± 0.06
Exp_4	0.92 ± 0.07	0.37 ± 0.05
Exp_5	0.90 ± 0.07	0.41 ± 0.08
Exp_6	0.89 ± 0.08	0.43 ± 0.08
Exp_7	0.91 ± 0.06	0.42 ± 0.07
Exp_8	0.90 ± 0.07	0.41 ± 0.09
Exp_9	0.92 ± 0.06	0.40 ± 0.07
Exp_10	0.91 ± 0.08	0.39 ± 0.09

下載: 導出CSV

表 3 針對主干網絡的比較實驗結果

Table 3 Comparison experiments by using different backbones

主干網絡	ResNet18	ResNet34	ResNet50	ResNet101	VGGNet
超聲圖像骨分割平均交并比IoU值	0.89 ± 0.10	0.90 ± 0.08	0.91 ± 0.07	0.89 ± 0.09	0.89 ± 0.09

下載: 導出CSV

表 4 本文所提出的超聲圖像分割網絡與其他最新的分割方法的實驗結果(IoU值)比較

Table 4 Comparison of the experimental results (IoU values) of the ultrasound image segmentation network proposed in this study with other state-of-the-art segmentation methods

實驗結果/ 方法	局部相位引導 CNN	BoneNet模型	濾波層引導 CNN	時空CNN	注意引導網絡 AGNet	三重注意力網絡 TriANet	本文方法
Exp_1	0.88	0.88	0.87	0.85	0.85	0.86	0.91
Exp_2	0.87	0.88	0.86	0.84	0.85	0.85	0.90
Exp_3	0.87	0.89	0.87	0.86	0.86	0.88	0.91
Exp_4	0.89	0.89	0.87	0.85	0.86	0.87	0.92
Exp_5	0.87	0.89	0.85	0.86	0.87	0.87	0.90
Exp_6	0.86	0.88	0.85	0.84	0.85	0.85	0.89
Exp_7	0.86	0.90	0.86	0.84	0.86	0.87	0.91
Exp_8	0.87	0.88	0.84	0.85	0.86	0.86	0.90
Exp_9	0.89	0.91	0.89	0.86	0.87	0.88	0.92
Exp_10	0.87	0.89	0.87	0.87	0.87	0.89	0.91
平均值	0.87 ± 0.13	0.88 ± 0.08	0.86 ± 0.14	0.87 ± 0.12	0.86 ± 0.09	0.88 ± 0.10	0.91 ± 0.07

下載: 導出CSV

<button id="qm3rj"><thead id="qm3rj"></thead></button>

亚洲第一网址_国产国产人精品视频69_久久久久精品视频_国产精品第九页

參考文獻(30)

[1]	Zhe Z, Zhu J J, Song F, He D W, Deng J Z, Chen F, et al. Intraoperative ultrasound-guided reduction of femoral shaft fractures using intramedullary nailing: A technical note. Archives of Orthopaedic and Trauma Surgery, 2019, 139(5): 589?596 doi: 10.1007/s00402-018-3085-8
[2]	Zhou H, Zhang G, Li M, Qu X Y, Cao Y J, Liu X, et al. Ultrasonography-guided closed reduction in the treatment of displaced transphyseal fracture of the distal humerus. Journal of Orthopaedic Surgery and Research, 2020, 15(1): Article No. 575 doi: 10.1186/s13018-020-02118-2
[3]	Wein W, Karamalis A, Baumgartner A, Navab N. Automatic bone detection and soft tissue aware ultrasound-CT registration for computer-aided orthopedic surgery. International Journal of Computer Assisted Radiology and Surgery, 2015, 10(6): 971?979 doi: 10.1007/s11548-015-1208-z
[4]	Hacihaliloglu I. Ultrasound imaging and segmentation of bone surfaces: A review. Technology, 2017, 5(2): 74?80 doi: 10.1142/S2339547817300049
[5]	Pandey P U, Quader N, Guy P, Garbi R, Hodgson A J. Ultrasound bone segmentation: A scoping review of techniques and validation practices. Ultrasound in Medicine and Biology, 2020, 46(4): 921?935 doi: 10.1016/j.ultrasmedbio.2019.12.014
[6]	Masson-Sibut A, Nakib A, Petit E, Leitner F. Computer-assisted intramedullary nailing using real-time bone detection in 2D ultrasound images. In: Proceedings of the 2nd International Workshop on Machine Learning in Medical Imaging. Toronto, Canada: Springer, 2011. 18?25
[7]	Wang P Y, Patel V M, Hacihaliloglu I. Simultaneous segmentation and classification of bone surfaces from ultrasound using a multi-feature guided CNN. In: Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada, Spain: Springer, 2018. 134?142
[8]	Huang Z X, Wang L W, Leung F H F, Banerjee S, Yang D, Lee T, et al. Bone feature segmentation in ultrasound spine image with robustness to speckle and regular occlusion noise. In: Proceedings of IEEE International Conference on Systems, Man, and Cybernetics (SMC). Toronto, Canada: IEEE, 2020. 1566?1571
[9]	Hacihaliloglu I, Abugharbieh R, Hodgson A, Rohling R. Bone segmentation and fracture detection in ultrasound using 3D local phase features. In: Proceedings of the 11th International Conference on Medical Image Computing and Computer-Assisted Intervention. New York, USA: Springer, 2008. 287?295
[10]	范家偉, 張如如, 陸萌, 何佳雯, 康霄陽, 柴文俊, 等. 深度學習方法在糖尿病視網膜病變診斷中的應用. 自動化學報, 2021, 47(5): 985?1004 Fan Jia-Wei, Zhang Ru-Ru, Lu Meng, He Jia-Wen, Kang Xiao-Yang, Chai Wen-Jun, et al. Applications of deep learning techniques for diabetic retinal diagnosis. Acta Automatica Sinica, 2021, 47(5): 985?1004
[11]	蔣蕓, 譚寧. 基于條件深度卷積生成對抗網絡的視網膜血管分割. 自動化學報, 2021, 47(1): 136?147 Jiang Yun, Tan Ning. Retinal vessel segmentation based on conditional deep convolutional generative adversarial networks. Acta Automatica Sinica, 2021, 47(1): 136?147
[12]	夏平, 施宇, 雷幫軍, 龔國強, 胡蓉, 師冬霞. 復小波域混合概率圖模型的超聲醫學圖像分割. 自動化學報, 2021, 47(1): 185?196 Xia Ping, Shi Yu, Lei Bang-Jun, Gong Guo-Qiang, Hu Rong, Shi Dong-Xia. Ultrasound medical image segmentation based on hybrid probabilistic graphical model in complex-wavelet domain. Acta Automatica Sinica, 2021, 47(1): 185?196
[13]	Ouahabi A, Taleb-Ahmed A. RETRACTED: Deep learning for real-time semantic segmentation: Application in ultrasound imaging. Pattern Recognition Letters, 2021, 144: 27?34 doi: 10.1016/j.patrec.2021.01.010
[14]	Baka N, Leenstra S, van Walsum T. Ultrasound aided vertebral level localization for lumbar surgery. IEEE Transactions on Medical Imaging, 2017, 36(10): 2138?2147 doi: 10.1109/TMI.2017.2738612
[15]	Ciganovic M, ?zdemir F, Farshad M, G?ksel O. Deep learning techniques for bone surface delineation in ultrasound. In: Proceedings of the SPIE 10955, Medical Imaging 2019: Ultrasonic Imaging and Tomography. San Diego, USA: SPIE, 2019. Article No. 109550Y
[16]	Wang P Y, Vives M, Patel V M, Hacihaliloglu I. Robust real-time bone surfaces segmentation from ultrasound using a local phase tensor-guided CNN. International Journal of Computer Assisted Radiology and Surgery, 2020, 15(7): 1127?1135 doi: 10.1007/s11548-020-02184-1
[17]	Alsinan A Z, Patel V M, Hacihaliloglu I. Automatic segmentation of bone surfaces from ultrasound using a filter-layer-guided CNN. International Journal of Computer Assisted Radiology and Surgery, 2019, 14(5): 775?783 doi: 10.1007/s11548-019-01934-0
[18]	Luan K, Li Z Y, Li J. An efficient end-to-end CNN for segmentation of bone surfaces from ultrasound. Computerized Medical Imaging and Graphics, 2020, 84: Article No. 101766 doi: 10.1016/j.compmedimag.2020.101766
[19]	Hu P, Caba F, Wang O, Lin Z, Sclaroff S, Perazzi F. Temporally distributed networks for fast video semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020. 8815?8824
[20]	Yao R, Xu X, Zhou Y, Zhao J Q, Fang L. Joint attention mechanism for unsupervised video object segmentation. In: Proceedings of the 4th Chinese Conference on Pattern Recognition and Computer Vision (PRCV). Beijing, China: Springer, 2021. 154?165
[21]	Alcázar J L, Bravo M A, Jeanneret G, Thabet A K, Brox T, Arbeláez P, et al. MAIN: Multi-attention instance network for video segmentation. Computer Vision and Image Understanding, 2021, 210: Article No. 103240 doi: 10.1016/j.cviu.2021.103240
[22]	He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 770?778
[23]	He K M, Zhang X Y, Ren S Q, Sun J. Identity mappings in deep residual networks. In: Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer, 2016. 630?645
[24]	Cox J, Rubin S, Adams J, Pereira C, Dighe M, Alessio A. Hyperparameter selection for ResNet classification of malignancy from thyroid ultrasound images. In: Proceedings of the SPIE 11314, Medical Imaging 2020: Computer-Aided Diagnosis. Houston, USA: SPIE, 2020. Article No. 1131447
[25]	Zhang Q, Cui Z P, Niu X G, Geng S J, Qiao Y. Image segmentation with pyramid dilated convolution based on ResNet and U-Net. In: Proceedings of the 24th International Conference on Neural Information Processing. Guangzhou, China: Springer, 2017. 364?372
[26]	Hacihaliloglu I, Rasoulian A, Rohling R N, Abolmaesumi P. Local phase tensor features for 3-D ultrasound to statistical shape+pose spine model registration. IEEE Transactions on Medical Imaging, 2014, 33(11): 2167?2179 doi: 10.1109/TMI.2014.2332571
[27]	Hacihaliloglu I. Localization of bone surfaces from ultrasound data using local phase information and signal transmission maps. In: Proceedings of the 5th International Workshop on Computational Methods and Clinical Applications in Musculoskeletal Imaging. Quebec City, Canada: Springer, 2017. 1?11
[28]	Xu K, Wen L Y, Li G R, Bo L F, Huang Q M. Spatiotemporal CNN for video object segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019. 1379?1388
[29]	Li J Y, Zhao Y K, Fu J, Wu J J, Liu J. Attention-guided network for semantic video segmentation. IEEE Access, 2019, 7: 140680?140689 doi: 10.1109/ACCESS.2019.2943365
[30]	Tian Y, Zhang Y J, Zhou D, Cheng G H, Chen W G, Wang R L. Triple attention network for video segmentation. Neurocomputing, 2020, 417: 202?211 doi: 10.1016/j.neucom.2020.07.078