1. <button id="qm3rj"><thead id="qm3rj"></thead></button>
      <samp id="qm3rj"></samp>
      <source id="qm3rj"><menu id="qm3rj"><pre id="qm3rj"></pre></menu></source>

      <video id="qm3rj"><code id="qm3rj"></code></video>

        1. <tt id="qm3rj"><track id="qm3rj"></track></tt>
            1. 2.765

              2022影響因子

              (CJCR)

              • 中文核心
              • EI
              • 中國科技核心
              • Scopus
              • CSCD
              • 英國科學文摘

              留言板

              尊敬的讀者、作者、審稿人, 關于本刊的投稿、審稿、編輯和出版的任何問題, 您可以本頁添加留言。我們將盡快給您答復。謝謝您的支持!

              姓名
              郵箱
              手機號碼
              標題
              留言內容
              驗證碼

              基于光流與多尺度上下文的圖像序列運動遮擋檢測

              馮誠 張聰炫 陳震 李兵 黎明

              馮誠, 張聰炫, 陳震, 李兵, 黎明. 基于光流與多尺度上下文的圖像序列運動遮擋檢測. 自動化學報, 2021, x(x): 1001?1012 doi: 10.16383/j.aas.c210324
              引用本文: 馮誠, 張聰炫, 陳震, 李兵, 黎明. 基于光流與多尺度上下文的圖像序列運動遮擋檢測. 自動化學報, 2021, x(x): 1001?1012 doi: 10.16383/j.aas.c210324
              Feng Cheng, Zhang Cong-Xuan, Chen Zhen, Li Bing, Li Ming. Occlusion detection based on optical flow and multiscale context aggregation. Acta Automatica Sinica, 2021, x(x): 1001?1012 doi: 10.16383/j.aas.c210324
              Citation: Feng Cheng, Zhang Cong-Xuan, Chen Zhen, Li Bing, Li Ming. Occlusion detection based on optical flow and multiscale context aggregation. Acta Automatica Sinica, 2021, x(x): 1001?1012 doi: 10.16383/j.aas.c210324

              基于光流與多尺度上下文的圖像序列運動遮擋檢測

              doi: 10.16383/j.aas.c210324
              基金項目: 國家重點研發計劃(2020YFC2003800), 國家自然科學基金(61866026, 61772255, 61866025), 江西省杰出青年人才計劃(20192BCB23011), 江西省自然科學基金重點項目(20202ACB214007), 江西省優勢科技創新團隊(20165BCB19007)資助
              詳細信息
                作者簡介:

                馮誠:南昌航空大學測試與光電工程學院碩士研究生. 主要研究方向為計算機視覺. E-mail: fengcheng00016@163.com

                張聰炫:南昌航空大學測試與光電工程學院副教授. 2014年獲得南京航空航天大學博士學位, 主要研究方向為圖像處理與計算機視覺. 本文通訊作者. E-mail: zcxdsg@163.com

                陳震:南昌航空大學測試與光電工程學院教授, 2003年獲得西北工業大學博士學位, 主要研究方向為圖像處理與計算機視覺. E-mail: dr_chenzhen@163.com

                李兵:中國科學院自動化研究所模式識別國家重點實驗室研究員, 2009年獲得北京交通大學博士學位, 主要研究方向為視頻內容理解與多媒體內容安全. E-mail: bli@nlpr.ia.ac.cn

                黎明:南昌航空大學信息工程學院教授, 1997年獲得南京航空航天大學博士學位, 主要研究方向為圖像處理與人工智能. E-mail: liming@nchu.edu.com

              Occlusion Detection Based on Optical Flow and Multiscale Context Aggregation

              Funds: Supported by National Key Research and Development Program of China (2020YFC2003800), National Natural Science Foundation of China (61866026, 61772255 and 61866025), Outstanding Young Scientist Project of Jiangxi Province (20192BCB23011), National Natural Science Foundation of Jiangxi Province (20202ACB214007) and Advantage Subject Team of Jiangxi Province (20165BCB19007)
              More Information
                Author Bio:

                FENG Cheng Master student at the School of Measuring and Optical Engineering, Nanchang Hangkong University, China. His main research interest is computer vision

                ZHANG Cong-Xuan Assistant Professor at School of Measuring and Optical Engineering, Nanchang Hangkong University, China. He received his Ph.D. degree from Nanjing University of Aeronautics and Astronautics in 2014. His research interest covers image processing and computer vision. Corresponding author of this paper

                CHEN Zhen Professor at School of Measuring and Optical Engineering, Nanchang Hangkong University, China. He received his Ph.D. degree from Northwestern Polytechnical University in 2003. His main research interest is image processing and computer vision

                LI Bing Professor at the National Key Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences. He received his Ph.D. degree from Beijing Jiaotong University in 2009. His research interest includes video understanding and multimedia content security

                Li Ming Professor at School of Information Engineering, Nanchang Hangkong University, China. He received the Ph.D. degree from Nanjing University of Aeronautics and Astronautics in 1997. His research interest covers image processing and artificial intelligence

              • 摘要: 針對非剛性運動和大位移場景下運動遮擋檢測的準確性與魯棒性問題, 本文提出一種基于光流與多尺度上下文的圖像序列運動遮擋檢測方法. 首先, 設計基于擴張卷積的多尺度上下文信息聚合網絡, 通過圖像序列多尺度上下文信息獲取更大范圍的圖像特征; 然后, 采用特征金字塔構建基于多尺度上下文與光流的端到端運動遮擋檢測網絡模型, 利用光流優化非剛性運動和大位移區域的運動遮擋信息; 最后, 構造基于運動邊緣的網絡模型訓練損失函數, 獲取準確的運動遮擋邊界. 分別采用MPI-Sintel和KITTI測試數據集對本文方法與現有的代表性遮擋檢測模型進行實驗對比與分析. 實驗結果表明, 本文方法能夠有效提高運動遮擋檢測的準確性, 尤其在非剛性運動和大位移等困難場景下具有更好的遮擋檢測魯棒性.
              • 圖  1  上下文網絡結構示意圖

                Fig.  1  Structure diagram of context network

                圖  2  常見的感受野擴張網絡結構示意圖

                (a) GoogLeNet (b) DeepLabv3+

                Fig.  2  The structure diagram of common receptive field expansion

                (a) GoogLeNet (b) DeepLabv3+

                圖  3  多尺度上下文信息聚合網絡結構示意圖

                Fig.  3  Structure diagram of multi-scale context information aggregation network

                圖  4  遮擋檢測網絡結構示意圖

                Fig.  4  Structure diagram of occlusion detection network

                圖  5  基于光流和多尺度上下文的遮擋檢測模型結構

                Fig.  5  The structure of the occlusion detection model based on optical flow and multi-scale context information

                圖  6  本文方法和IRR-PWC方法遮擋檢測結果對比

                Fig.  6  Comparison of occlusion detection results between our method and IRR-PWC method

                圖  7  MPI-Sinte數據集非剛性運動與大位移序列遮擋檢測對比圖. 從左往右:alley_2、ambush_2、market_6以及temple_2序列.

                Fig.  7  Comparison results of occlusion detection between non-rigid motion and large-displacement sequences on MPI-Sinte dataset. From left to right are alley_2、ambush_2、market_6 and temple_2 sequence.

                圖  8  各個遮擋檢測方法在KITTI數據集上的遮擋檢測結果對比圖. 從左往右分別是輸入圖像和Unflow、Back2Future、MaskFlownet、IRR-PWC以及本文方法的運動遮擋檢測圖.

                Fig.  8  Comparison of occlusion detection results of each occlusion detection method on KITTI dataset. From left to right are the input image, Unflow, back2future, MaskFlownet, IRR-PWC and our method.

                圖  9  利用光流真實值生成的運動遮擋掩膜部分示例圖 (N=3)

                Fig.  9  Examples of motion boundary mask generated by ground truth of flow field (N=3)

                圖  10  各消融模型可視化結果對比圖

                Fig.  10  Comparison of visualization results of each ablation model

                表  1  MPI-Sintel數據集平均F1分數對比結果

                Table  1  Comparison of Average F1 score on MPI-Sintel dataset

                對比方法多幀類型cleanfinal
                Unflow[24]傳統方法0.280.27
                Back2Future[25]無監督學習0.490.44
                MaskFlownet[27]無監督學習0.370.36
                IRR-PWC[26]監督學習0.710.67
                本文方法監督學習0.750.72
                下載: 導出CSV

                表  2  MPI-Sintel數據集平均漏檢與誤檢率對比結果

                Table  2  Comparison of average omission rate and false rate on MPI-Sintel dataset

                對比方法Clean Final
                ORFRORFR
                Unflow[24]1.96%18.32% 1.94%20.51%
                Back2Future[25]5.03%2.75%5.08%2.96%
                MaskFlownet[27]5.77%1.37%5.76%1.72%
                IRR-PWC[26]1.98%0.96%2.84%1.29%
                本文方法1.85%0.83%2.31%1.08%
                下載: 導出CSV

                表  3  非剛性運動與大位移圖像序列運動遮擋檢測平均F1分數對比結果

                Table  3  Comparison of average F1 scores of occlusion detection between non-rigid motion and large-displacement image sequences

                對比方法
                clean final
                alley_2ambush_2market_6temple_2alley_2ambush_2market_6temple_2
                Unflow[24]0.41490.43130.43300.3243 0.40570.39200.44990.3120
                Back2Future[25]0.68160.58880.62900.27120.67560.51990.62390.2683
                MaskFlownet[27]0.50570.54030.46600.38380.50390.40850.47350.3508
                IRR-PWC[26]0.87090.91720.81550.74040.87700.78090.80230.6905
                本文方法0.88110.92160.83040.77470.87640.79590.81060.7103
                下載: 導出CSV

                表  4  不同方法的時間消耗對比(加粗為評價最優值)

                Table  4  Comparison of time consumption of different methods (bold is the best evaluation value)

                對比方法多幀輸入類型運行時間
                Unflow[24]傳統方法0.13 s
                Back2Future[25]無監督學習0.13 s
                MaskFlownet[27]無監督學習0.10 s
                IRR-PWC[26]監督學習0.18 s
                本文方法監督學習0.19 s
                下載: 導出CSV

                表  5  MPI-Sintel全序列平均F1分數對比(加粗為評價最優值)

                Table  5  Comparison of average F1 scores of whole image sequence on MPI-Sintel (bold is the best evaluation value)

                模型類型
                MPI-Sintel training dataset
                cleanfinal運行時間訓練時間
                全模型0.750.720.19 s13days
                去除多尺度上下文網絡0.720.680.18 s12days
                去除邊緣損失函數0.740.710.19 s13days
                下載: 導出CSV

                表  6  MPI-Sintel全序列在不同運動邊界區域內的平均F1分數對比(加粗為評價最優值)

                Table  6  Comparison of average F1 scores of whole image sequence in different motion boundary regions on MPI-Sintel (bold is the best evaluation value)

                模型類型MPI-Sintel training dataset
                clean final
                N=1N=3N=5N=10N=1N=3N=5N=10
                全模型0.630.670.690.71 0.590.620.640.67
                去除多尺度上下文網絡0.590.620.650.670.550.590.610.63
                去除邊緣損失函數0.600.640.670.690.560.600.620.64
                下載: 導出CSV
                1. <button id="qm3rj"><thead id="qm3rj"></thead></button>
                  <samp id="qm3rj"></samp>
                  <source id="qm3rj"><menu id="qm3rj"><pre id="qm3rj"></pre></menu></source>

                  <video id="qm3rj"><code id="qm3rj"></code></video>

                    1. <tt id="qm3rj"><track id="qm3rj"></track></tt>
                        亚洲第一网址_国产国产人精品视频69_久久久久精品视频_国产精品第九页
                      1. [1] 張世輝, 何琦, 董利健, 杜雪哲. 基于遮擋區域建模和目標運動估計的動態遮擋規避方法. 自動化學報, 2019, 45(4): 771-786.

                        Zhang Shi-Hui, He Qi, Dong Li-Jian, Du Xue-Zhe. Dynamic occlusion avoidance approach by means of occlusion region model and object motion estimation. Acta Automatica Sinica, 2019, 45(4): 771-786.
                        [2] Yu C, Bo Y, Bo W, Yan W D, Robby T. Occlusion-aware networks for 3D human pose estimation in video. In: Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea: IEEE, 2019.723−732.
                        [3] 張聰炫, 陳震, 熊帆, 黎明, 葛利躍, 陳昊. 非剛性稠密匹配大位移運動光流估計. 電子學報, 2019, 47(6): 1316-1323. doi: 10.3969/j.issn.0372-2112.2019.06.019

                        Zhang Cong-Xuan, Chen Zhen, Xiong Fan, Li Ming, Ge Li-Yue, Chen Hao. Large displacement motion optical flow estimation with non-rigid dense patch matching. Acta Electronica Sinica. 2019, 47(6): 1316-1323. doi: 10.3969/j.issn.0372-2112.2019.06.019
                        [4] 姚乃明, 郭清沛, 喬逢春, 陳輝, 王宏安. 基于生成式對抗網絡的魯棒人臉表情識別. 自動化學報, 2018, 44(5): 865-877.

                        Yao Nai-Ming, Guo Qing-Pei, Qiao Feng-Chun, Chen Hui, Wang Hong-An. Robust facial expression recognition with generative adversarial networks. Acta Automatica Sinica, 2019, 44(5): 865-877.
                        [5] Pan J Y, Bo H. Robust occlusion handling in object tracking. In: Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Minneapolis, MN, USA: IEEE, 2007.1?8.
                        [6] 劉鑫, 許華榮, 胡占義. 基于 GPU 和 Kinect 的快速物體重建. 自動化學報, 2012, 38(8): 1288-1297.

                        Liu Xin, Xu Hua-Rong, Hu Zhan-Yi. GPU based fast 3D-object modeling with Kinect. 自動化學報, 2012, 38(8): 1288-1297
                        [7] 張聰炫, 陳震, 黎明. 單目圖像序列光流三維重建技術研究綜述. 電子學報, 2016, 44(12): 3044-3052. doi: 10.3969/j.issn.0372-2112.2016.12.033

                        Zhang Cong-Xuan, Chen Zhen, Li Ming, Review of the 3D reconstruction technology based on optical flow of monocular image sequence. Acta Electronica Sinica. 2016, 44 (12): 3044-3052. doi: 10.3969/j.issn.0372-2112.2016.12.033
                        [8] Bailer C, Taetz B, Stricker D. Flow Fields: Dense correspondence fields for highly accurate large displacement optical flow estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(8): 1879-1892. doi: 10.1109/TPAMI.2018.2859970
                        [9] Wolf L, Gadot D. PatchBatch: A batch augmented loss for optical flow. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA: IEEE, 2016.4236?4245.
                        [10] Li Y S, Song R, Hu Y L. Efficient coarse-to-fine patch match for large displacement optical flow. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA: IEEE, 2016.5704?5712.
                        [11] Menze M, Heipke C, Geiger A. Discrete optimization for optical flow. In: Proceedings of the 2015 German Conference on Pattern Recognition (GCPR), Aachen, Germany: Springer Press, 2015.16?28.
                        [12] Chen Q F, Koltun V. Full flow: Optical flow estimation by global optimization over regular grids. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA: IEEE, 2016.4706?4714.
                        [13] Guney F, Geiger A. Deep discrete flow. In: Proceedings of the 2016 Asian Conference on Computer Vision (ACCV), Taipei, Taiwan, China: Springer Press, 2016.207?224.
                        [14] Hur J, Roth S. Joint optical flow and temporally consistent semantic segmentation. In: Proceedings of the 2016 European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands: Springer, 2016.163?177.
                        [15] Ince S, Konrad J. Occlusion-aware optical flow estimation. IEEE Transactions on Image Process, 2008, 17(8): 1443-1451. doi: 10.1109/TIP.2008.925381
                        [16] Sun D Q, Liu C, Pfister H. Local layering for joint motion estimation and occlusion detection. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, Ohio, USA: IEEE, 2014.1098?1105.
                        [17] Sun D Q, Sudderth E B, Black M J. Layered image motion with explicit occlusions, temporal consistency, and depth ordering. In: Proceedings of the 24st International Conference on Neural Information Processing Systems (NIPS), Vancouver, Canada: Curran Associates Inc., 2010.2226?2234.
                        [18] Vogel C, Roth S, Schindler K. View-consistent 3D scene flow estimation over multiple frames. In: Proceedings of the 2014 European Conference on Computer Vision (ECCV), Zurich, Switzerland: Springer Press, 2014.263?278.
                        [19] Zanfir A, Sminchisescu C. Large displacement 3D scene flow with occlusion reasoning. In: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile: IEEE, 2015.4417?4425.
                        [20] Zhang C X, Chen Z, Wang M R, Li M, Jiang S F. Robust non-local TV-L1 optical flow estimation with occlusion detection. IEEE Transactions on Image Process, 2017, 26(8): 4055-4067. doi: 10.1109/TIP.2017.2712279
                        [21] 張聰炫, 陳震, 汪明潤, 黎明, 江少鋒. 基于光流與Delaunay三角網格的圖像序列運動遮擋檢測. 電子學報, 2018, 46(2): 479-485. doi: 10.3969/j.issn.0372-2112.2018.02.030

                        Zhang Cong-Xuan, Chen Zhen, Wang Ming-Run, Li Ming, Jiang Shao-Feng. Motion occlusion detecting from image sequence based on optical flow and Delaunay triangulation. Acta Electronica Sinica, 2018, 46(2): 479-485. doi: 10.3969/j.issn.0372-2112.2018.02.030
                        [22] Kennedy R, Taylor C J. Optical flow with geometric occlusion estimation and fusion of multiple frames. In: International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR), Hong Kong, China: IEEE, 2015.364?377.
                        [23] Yu J J, Harley A W, Derpanis K G. Back to basics: Unsupervised learning of optical flow via brightness constancy and motion smoothness. In: Proceedings of the 2016 European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands: Springer, 2016.3?10.
                        [24] Meister S, Hur J, Roth S. UnFlow: Unsupervised learning of optical flow with a bidirectional census loss. In: Proceedings of the 31rd AAAI Conference on Artificial Intelligence (AAAI), San Francisco, California, USA: AAAI, 2017.7251?7259.
                        [25] Janai J, Güney F, Ranjan A, Black M, Geiger A. Unsupervised learning of multi-frame optical flow with occlusions. In: Proceedings of the 2018 European Conference on Computer Vision (ECCV), Munich, Germany: Springer, 2018.713?731.
                        [26] Hur J, Roth S. Iterative residual refinement for joint optical flow and occlusion estimation. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA: IEEE, 2019.5747?5756.
                        [27] Zhao S Y, Sheng Y L, Dong Y, Chang E I C, Xu Y. MaskFlownet: Asymmetric feature matching with learnable occlusion mask. In: Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual: IEEE, 2020.6277?6286.
                        [28] Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. In: Proceedings of the 2015 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Boston, Massachusetts, USA: IEEE, 2015.1?9.
                        [29] Chen L C, Zhu Y K, Papandreou G, Schroff F, Adam H. Encoder-Decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the 2018 European Conference on Computer Vision (ECCV), Munich, Germany: Springer, 2018.833?851.
                        [30] Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions. [Online], available: https://arxiv.org/abs/1511.07122, Apr 30, 2016.
                        [31] Yang M K, Yu K, Zhang C, Li Z W, Yang K Y. DenseASPP for semantic segmentation in street scenes. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, Utah, USA: IEEE, 2018.3684?3692.
                        [32] Mehta S, Rastegari M, Caspi A, Shapiro L, Hajishirzi H. ESPNet: Efficient spatial pyramid of dilated convolutions for semantic segmentation. In: Proceedings of the 2018 European Conference on Computer Vision (ECCV), Munich, Germany: Springer, 2018.561?580.
                        [33] Butler D J, Wulff J, Stanley G B, Black M J. A naturalistic open source movie for optical flow evaluation. In: Proceedings of the 2012 European Conference on Computer Vision (ECCV), Florence, Italy: Springer, 2012.611?625.
                        [34] Menze M, Geiger A. Object scene flow for autonomous vehicles. In: Proceedings of the 2015 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Boston, Massachusetts, USA: IEEE, 2015.2061?3070.
                      2. 加載中
                      3. 計量
                        • 文章訪問數:  677
                        • HTML全文瀏覽量:  358
                        • 被引次數: 0
                        出版歷程
                        • 收稿日期:  2021-04-15
                        • 錄用日期:  2021-07-02
                        • 網絡出版日期:  2021-08-31

                        目錄

                          /

                          返回文章
                          返回