1. <button id="qm3rj"><thead id="qm3rj"></thead></button>
      <samp id="qm3rj"></samp>
      <source id="qm3rj"><menu id="qm3rj"><pre id="qm3rj"></pre></menu></source>

      <video id="qm3rj"><code id="qm3rj"></code></video>

        1. <tt id="qm3rj"><track id="qm3rj"></track></tt>
            1. 2.765

              2022影響因子

              (CJCR)

              • 中文核心
              • EI
              • 中國科技核心
              • Scopus
              • CSCD
              • 英國科學文摘

              留言板

              尊敬的讀者、作者、審稿人, 關于本刊的投稿、審稿、編輯和出版的任何問題, 您可以本頁添加留言。我們將盡快給您答復。謝謝您的支持!

              姓名
              郵箱
              手機號碼
              標題
              留言內容
              驗證碼

              基于閱讀技巧識別和雙通道融合機制的機器閱讀理解方法

              彭偉 胡玥 李運鵬 謝玉強 牛晨旭

              彭偉, 胡玥, 李運鵬, 謝玉強, 牛晨旭. 基于閱讀技巧識別和雙通道融合機制的機器閱讀理解方法. 自動化學報, 2024, 50(5): 958?969 doi: 10.16383/j.aas.c220983
              引用本文: 彭偉, 胡玥, 李運鵬, 謝玉強, 牛晨旭. 基于閱讀技巧識別和雙通道融合機制的機器閱讀理解方法. 自動化學報, 2024, 50(5): 958?969 doi: 10.16383/j.aas.c220983
              Peng Wei, Hu Yue, Li Yun-Peng, Xie Yu-Qiang, Niu Chen-Xu. A machine reading comprehension approach based on reading skill recognition and dual channel fusion mechanism. Acta Automatica Sinica, 2024, 50(5): 958?969 doi: 10.16383/j.aas.c220983
              Citation: Peng Wei, Hu Yue, Li Yun-Peng, Xie Yu-Qiang, Niu Chen-Xu. A machine reading comprehension approach based on reading skill recognition and dual channel fusion mechanism. Acta Automatica Sinica, 2024, 50(5): 958?969 doi: 10.16383/j.aas.c220983

              基于閱讀技巧識別和雙通道融合機制的機器閱讀理解方法

              doi: 10.16383/j.aas.c220983
              基金項目: 國家自然科學基金(62006222, U21B2009), 中國科學院戰略性先導研究計劃(XDC02030400)資助
              詳細信息
                作者簡介:

                彭偉:中關村實驗室助理研究員. 2023年獲得中國科學院信息工程研究所博士學位. 主要研究方向為對話生成, 網絡空間安全. E-mail: pengwei@iie.ac.cn

                胡玥:中國科學院信息工程研究所研究員. 主要研究方向為自然語言處理, 人工智能. 本文通信作者. E-mail: huyue@iie.ac.cn

                李運鵬:中國科學院信息工程研究所博士研究生. 2019年獲得山東大學學士學位. 主要研究方向為自然語言處理. E-mail: liyunpeng@iie.ac.cn

                謝玉強:2023年獲得中國科學院大學博士學位. 主要研究方向為自然語言處理, 認知建模. E-mail: yuqiang.xie@kunlun-inc.com

                牛晨旭:中國科學院信息工程研究所博士研究生. 2021年獲得西安電子科技大學學士學位. 主要研究方向為自然語言處理. E-mail: niuchenxu@iie.ac.cn

              A Machine Reading Comprehension Approach Based on Reading Skill Recognition and Dual Channel Fusion Mechanism

              Funds: Supported by National Natural Science Foundation of China (62006222, U21B2009) and Strategic Priority Research Program of Chinese Academy of Science (XDC02030400)
              More Information
                Author Bio:

                PENG Wei Assistant professor at Zhongguancun Laboratory. He received his Ph.D. degree from Institute of Information Engineering, Chinese Academy of Sciences in 2023. His research interest covers dialog generation and cyber security

                HU Yue Professor at the Institute of Information Engineering, Chinese Academy of Sciences. Her research interest covers natural language processing and artificial intelligence. Corresponding author of this paper

                LI Yun-Peng Ph.D. candidate at the Institute of Information Engineering, Chinese Academy of Sciences. He received his bachelor degree from Shandong University in 2019. His main research interest is natural language processing

                XIE Yu-Qiang He received his Ph.D. degree from University of Chinese Academy of Sciences in 2023. His research interest covers natural language processing and cognitive modeling

                NIU Chen-Xu Ph.D. candidate at the Institute of Information Engineering, Chinese Academy of Sciences. She received her bachelor degree from Xidian University in 2021. Her main research interest is natural language processing

              • 摘要: 機器閱讀理解任務旨在要求系統對給定文章進行理解, 然后對給定問題進行回答. 先前的工作重點聚焦在問題和文章間的交互信息, 忽略了對問題進行更加細粒度的分析(如問題所考察的閱讀技巧是什么?). 受先前研究的啟發, 人類對于問題的理解是一個多維度的過程. 首先, 人類需要理解問題的上下文信息; 然后, 針對不同類型問題, 識別其需要使用的閱讀技巧; 最后, 通過與文章交互回答出問題答案. 針對這些問題, 提出一種基于閱讀技巧識別和雙通道融合的機器閱讀理解方法, 對問題進行更加細致的分析, 從而提高模型回答問題的準確性. 閱讀技巧識別器通過對比學習的方法, 能夠顯式地捕獲閱讀技巧的語義信息. 雙通道融合機制將問題與文章的交互信息和閱讀技巧的語義信息進行深層次的融合, 從而達到輔助系統理解問題和文章的目的. 為了驗證該模型的效果, 在FairytaleQA數據集上進行實驗, 實驗結果表明, 該方法實現了在機器閱讀理解任務和閱讀技巧識別任務上的最好效果.
              • 圖  1  在FairytaleQA數據集中的一個例子

                Fig.  1  An example in FairytaleQA dataset

                圖  2  本文模型總體結構

                Fig.  2  Overall structure of our model

                圖  3  閱讀技巧識別器結構

                Fig.  3  Structure of the reading skill recognizer

                圖  4  雙通道融合機制結構圖

                Fig.  4  Structure of dual channel fusion machanism

                圖  5  雙通道融合機制的性能比較

                Fig.  5  The performances comparison on the dual channel fusion mechanism

                圖  6  3種不同融合機制的比較

                Fig.  6  Comparison of the three different fusion mechanisms

                圖  7  閱讀技巧識別的可視化

                Fig.  7  The visualization of the reading skill recognition

                表  1  FairytaleQA數據集的主要統計數據

                Table  1  Core statistics of the FairytaleQA dataset

                項目均值標準偏差最小值最大值
                每個故事章節數15.69.8260
                每個故事單詞數2305.41480.82287577
                每個章節單詞數147.760.012447
                每個故事問題數41.729.15161
                每個章節問題數2.92.4018
                每個問題單詞數10.53.2327
                每個答案單詞數7.25.8170
                下載: 導出CSV

                表  2  FairytaleQA數據集中驗證集和測試集上的性能對比 (%)

                Table  2  Performance comparison on the validation and the test set in FairytaleQA dataset (%)

                模型名稱驗證集測試集
                B-1B-2B-3B-4ROUGE-LMETEORB-1B-2B-3B-4ROUGE-LMETEOR
                輕量化模型
                Seq2Seq25.126.672.010.8113.616.9426.336.722.170.8114.557.34
                CAQA-LSTM28.058.243.661.5716.158.1130.048.854.171.9817.338.60
                Transformer21.874.941.530.5910.326.0121.725.211.740.6710.276.22
                預訓練語言模型
                DistilBERT9.708.20
                BERT10.409.70
                BART19.137.923.422.1412.256.5121.058.933.902.5212.666.70
                微調模型
                BART-Question-types49.10
                CAQA-BART52.5944.1742.7640.0753.2028.3155.7347.0043.6840.4555.1328.80
                BART-NarrativeQA45.3439.1736.3334.1047.3924.6548.1341.5038.2636.9749.1626.93
                BART-FairytaleQA$ \dagger $51.7443.3041.2338.2953.8827.0954.0445.9842.0839.4653.6427.45
                BART-FairytaleQA ?51.2843.9641.5139.0554.1126.8654.8246.3743.0239.7154.4427.82
                本文模型54.2147.3844.6543.0258.9929.7057.3649.5546.2342.9158.4830.93
                人類表現65.1064.40
                下載: 導出CSV

                表  3  FairytaleQA數據集中驗證集和測試集上的各組件消融實驗結果 (%)

                Table  3  The performance of ablation study on each component in our model on the validation set and the test set of the FairytaleQA dataset (%)

                模型設置驗證集測試集
                B-1B-2B-3B-4ROUGE-LMETEORB-1B-2B-3B-4ROUGE-LMETEOR
                SOTA 模型51.2843.9641.5139.0554.1126.8654.8246.3743.0239.7154.4427.82
                去除閱讀技巧識別器52.1544.4742.1140.7355.3827.4554.9047.1643.5540.6756.4829.31
                去除對比學習損失53.2045.0742.8841.9456.7528.1555.2247.9844.1341.4257.3430.20
                去除雙通道融合機制52.5845.3843.1541.6257.2227.7555.7948.2044.9641.2857.1229.88
                本文模型54.2147.3844.6543.0258.9929.7057.3649.5546.2342.9158.4830.93
                下載: 導出CSV

                表  4  基于交叉熵損失的方法和基于有監督對比學習的方法在2個任務上的效果 (%)

                Table  4  The performance of cross-entropy-loss-based method and supervised contrastive learning method on the two tasks (%)

                實驗設置準確率B-4ROUGE-LMETEOR
                基于交叉熵損失的方法91.4041.4257.3430.20
                本文基于有監督對比
                學習損失的方法
                93.7742.9158.4830.93
                下載: 導出CSV

                表  5  不同輸入下的閱讀技巧識別器的識別準確率 (%)

                Table  5  The recognition accuracy of reading skill recognizer under different inputs (%)

                實驗設置驗證集測試集
                只輸入問題85.3182.56
                輸入問題和文章92.2493.77
                下載: 導出CSV
                1. <button id="qm3rj"><thead id="qm3rj"></thead></button>
                  <samp id="qm3rj"></samp>
                  <source id="qm3rj"><menu id="qm3rj"><pre id="qm3rj"></pre></menu></source>

                  <video id="qm3rj"><code id="qm3rj"></code></video>

                    1. <tt id="qm3rj"><track id="qm3rj"></track></tt>
                        亚洲第一网址_国产国产人精品视频69_久久久久精品视频_国产精品第九页
                      1. [1] Hermann K M, Kocisky T, Grefenstette E, Espeholt L, Kay W, Suleyman M, et al. Teaching machines to read and comprehend. In: Proceedings of the Neural Information Processing Systems. Montreal, Canada: 2015. 1693–1701
                        [2] Seo M J, Kembhavi A, Farhadi A, Hajishirzi H. Bidirectional attention flow for machine comprehension. arXiv preprint arXiv: 1611.01603, 2016.
                        [3] Tay Y, Wang S, Luu A T, Fu J, Phan M C, Yuan X, et al. Simple and effective curriculum pointer-generator networks for reading comprehension over long narratives. In: Proceedings of the Conference of the Association for Computational Linguistics. Florence, Italy: 2019. 4922–4931
                        [4] Peng W, Hu Y, Yu J, Xing L X, Xie Y Q. APER: Adaptive evidence-driven reasoning network for machine reading comprehension with unanswerable questions. Knowledge-Based Systems, 2021, 229: Article No. 107364 doi: 10.1016/j.knosys.2021.107364
                        [5] Perevalov A, Both A, Diefenbach D, Ngomo A N. Can machine translation be a reasonable alternative for multilingual question answering systems over knowledge graphs? In: Proceedings of the ACM Web Conference. Lyon, France: 2022. 977–986
                        [6] Xu Y, Wang D, Yu M, Ritchie D, Yao B, Wu T, et al. Fantastic questions and where to find them: FairytaleQA——An authentic dataset for narrative comprehension. In: Proceedings of the Conference of the Association for Computational Linguistics. Dublin, Ireland: 2022. 447–460
                        [7] Liu S, Zhang X, Zhang S, Wang H, Zhang W. Neural machine reading comprehension: Methods and trends. arXiv preprint arXiv: 1907.01118, 2019.
                        [8] Yan M, Xia J, Wu C, Bi B, Zhao Z, Zhang J, et al. A deep cascade model for multi-document reading comprehension. In: Proceedings of the Conference on Artificial Intelligence. Honolulu, USA: 2019. 7354–7361
                        [9] Liao J, Zhao X, Li X, Tang J, Ge B. Contrastive heterogeneous graphs learning for multi-hop machine reading comprehension. World Wide Web, 2022, 25(3): 1469?1487 doi: 10.1007/s11280-021-00980-6
                        [10] Lehnert W G. Human and computational question answering. Cognitive Science, 1977, 1(1): 47?73 doi: 10.1207/s15516709cog0101_3
                        [11] Kim Y. Why the simple view of reading is not simplistic: Unpacking component skills of reading using a direct and indirect effect model of reading. Scientific Studies of Reading, 2017, 21(4): 310?333 doi: 10.1080/10888438.2017.1291643
                        [12] Sugawara S, Yokono H, Aizawa A. Prerequisite skills for reading comprehension: Multi-perspective analysis of MCTest datasets and systems. In: Proceedings of the Conference on Artificial Intelligence. San Francisco, USA: 2017. 3089–3096
                        [13] Weston J, Bordes A, Chopra S, Mikolov T. Towards AI-complete question answering: A set of prerequisite toy tasks. arXiv preprint arXiv: 1502.05698, 2015.
                        [14] Purves A C, S?ter A, Takala S, V?h?passi A. Towards a domain-referenced system for classifying composition assignments. Research in the Teaching of English, 1984: 385?416
                        [15] V?h?passi A. On the specification of the domain of school writing. Afinlan Vuosikirja, 1981: 85?107
                        [16] Chen D, Bolton J, Manning C D. A thorough examination of the CNN/daily mail reading comprehension task. arXiv preprint arXiv: 1606.02858, 2016.
                        [17] Rajpurkar P, Zhang J, Lopyrev K, Liang P. Squad: 100,000 + questions for machine comprehension of text. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Austin, USA: 2016. 2383–2392
                        [18] Richardson M, Renshaw E. MCTest: A challenge dataset for the open-domain machine comprehension of text. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Washington, USA: 2013. 193–203
                        [19] Kocisk'y T, Schwarz J, Blunsom P, Dyer C, Hermann K M, Melis G. The narrativeqa reading comprehension challenge. Transactions of the Association for Computational Linguistics, 2018, 6: 317?328 doi: 10.1162/tacl_a_00023
                        [20] Yang B, Mitchell T M. Leveraging knowledge bases in LSTMs for improving machine reading. In: Proceedings of the Conference of the Association for Computational Linguistics. Vancou-ver, Canada: 2017. 1436–1446
                        [21] Zhang Z, Wu Y, Zhou J, Duan S, Zhao H, Wang R. SG-Net: Syntax-guided machine reading comprehension. In: Proceedings of the Conference on Artificial Intelligence. New York, USA: 2020. 9636–9643
                        [22] Kao K Y, Chang C H. Applying information extraction to storybook question and answer generation. In: Proceedings of the Conference on Computational Linguistics and Speech Process-ing. Taipei, China: 2022. 289–298
                        [23] Lu J, Sun X, Li B, Bo L, Zhang T. BEAT: Considering question types for bug question answering via templates. Knowledge-Based Systems, 2021, 225: Article No. 107098
                        [24] Yang C, Jiang M, Jiang B, Zhou W, Li K. Co-attention network with question type for visual question answering. IEEE Access, 2019, (7): 40771?40781 doi: 10.1109/ACCESS.2019.2908035
                        [25] Wu Z, Xiong Y, Yu S X, Lin D. Unsupervised feature learning via non-parametric instance discrimination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Computer Society, 2018. 3733– 3742
                        [26] Chen X, He K. Exploring simple siamese representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Virtual Event: IEEE, 2021. 15750– 15758
                        [27] Yang J, Duan J, Tran S, Xu Y, Chanda S, Li Q C, et al. Vision-language pre-training with triple contrastive learning. arXiv preprint arXiv: 2202.10401, 2022.
                        [28] Chopra S, Hadsell R, LeCun Y. Learning a similarity metric discriminatively, with application to face verification. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, USA: IEEE Computer Society, 2005. 539–546
                        [29] Weinberger K Q, Saul L K. Distance metric learning for large margin nearest neighbor classification. Journal of Machine Lear-ning Research, 2009, 10(2): 207?244 doi: 10.5555/1577069.1577078
                        [30] Gao T, Yao X, Chen D. SimCSE: Simple contrastive learning of sentence embeddings. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Virtual Event: 2021. 6894–6910
                        [31] Giorgi J M, Nitski O, Wang B, Bader G D. DeCLUTR: Deep contrastive learning for unsupervised textual representations. In: Proceedings of the Conference of the Association for Computational Linguistics. Virtual Event: 2021. 879–895
                        [32] Khosla P, Teterwak P, Wang C, Sarna A, Tian Y, Krishnan D, et al. Supervised contrastive learning. In: Proceedings of the Neural Information Processing Systems. Virtual Event: 2020. 18661–18673
                        [33] Li S, Hu X, Lin L, Wen L. Pair-level supervised contrastive learning for natural language inference. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. Virtual Event: IEEE, 2022. 8237–8241
                        [34] Devlin J, Chang M, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of North American Chapter of the Association for Computational Linguistics. Minneapolis, USA: 2019. 4171– 4186
                        [35] Papineni K, Roukos S, Ward T, Zhu W. BLEU: A method for automatic evaluation of machine translation. In: Proceedings of the Conference of the Association for Computational Linguistics. Philadelphia, USA: 2002. 311–318
                        [36] Lin C Y. ROUGE: A package for automatic evaluation of summaries. Text Summarization Branches Out, 2004: 74?81
                        [37] Banerjee S, Lavie A. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the Conference of the Association for Computational Linguistics. Ann Arbor, USA: 2005. 65–72
                        [38] Lewis M, Liu Y, Goyal N, Ghazvininejad M, Mohamed A, Levy O, et al. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of the Association for Computational Linguistics. Virtual Event: 2020. 7871–7880
                        [39] Loshchilov I, Hutter F. Fixing weight decay regularization in adam. arXiv preprint arXiv: 1711.05101, 2017.
                        [40] Cho K, Merrienboer B, Bengio Y, Gulcehre C, Bahdanau D, Bougares F, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the Empirical Methods in Natural Language Processing. Doha, Qatar: 2014. 1724–1734
                        [41] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A, et al. Attention is all you need. In: Proceedings of the Neural Information Processing Systems. Long Beach, USA: 2017. 5998–6008
                        [42] Sanh V, Debut L, Chaumond J, Wolf T. DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv preprint arXiv: 1910.01108, 2019.
                        [43] Mou X, Yang C, Yu M, Yao B, Guo X, Potdar S. Narrative question answering with cutting-edge open-domain QA techni-ques: A comprehensive study. Transactions of the Association for Computational Linguistics, 2021, 9: 1032?1046 doi: 10.1162/tacl_a_00411
                      2. 加載中
                      3. 圖(7) / 表(5)
                        計量
                        • 文章訪問數:  223
                        • HTML全文瀏覽量:  127
                        • PDF下載量:  44
                        • 被引次數: 0
                        出版歷程
                        • 收稿日期:  2022-12-20
                        • 網絡出版日期:  2023-11-08
                        • 刊出日期:  2024-05-20

                        目錄

                          /

                          返回文章
                          返回