基于大語(yǔ)言模型的復雜任務(wù)自主規劃處理框架
doi: 10.16383/j.aas.c240088
-
1.
國防科技大學(xué)系統工程學(xué)院 長(cháng)沙 410073
-
2.
暨南大學(xué)公共/應急管理學(xué)院 廣州 510632
-
3.
中國科學(xué)院自動(dòng)化研究所復雜系統管理與控制國家重點(diǎn)實(shí)驗室 北京 100190
-
4.
青島智能產(chǎn)業(yè)技術(shù)研究院 青島 266000
基金項目: 國家自然科學(xué)基金(62103420, 62103425, 62103428, 62306329), 湖南省自然科學(xué)基金(2023JJ40676, 2021JJ40697, 2021JJ40702), 國防科技大學(xué)青年自主創(chuàng )新基金(ZK-2023-31)資助
Autonomous Planning and Processing Framework for Complex Tasks Based on Large Language Models
-
1.
College of Systems Engineering, National University of Defense Technology, Changsha 410073
-
2.
School of Public Management/Emergency Management, Jinan University, Guangzhou 510632
-
3.
The State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190
-
4.
Qingdao Academy of Intelligent Industries, Qingdao 266000
Funds: Supported by National Natural Science Foundation of China (62103420, 62103425, 62103428, 62306329), Natural Science Foundation of Hunan Province (2023JJ40676, 2021JJ40697, 2021JJ40702), and Youth Independent Innovation Fundation of National University of Defense Technology (ZK-2023-31)
-
摘要: 隨著(zhù)深度學(xué)習和自然語(yǔ)言處理技術(shù)的進(jìn)步, 大語(yǔ)言模型(Large language models, LLMs)展現出巨大潛力. 盡管如此, 它們在處理復雜任務(wù)時(shí)仍存在局限性, 特別是在任務(wù)需要結合規劃及外部工具調用的場(chǎng)合. 面向這一挑戰, 提出國內首個(gè)以軍事游戲為背景的中文的復雜任務(wù)規劃與執行數據集(Complex task planning and execution dataset, CTPaE), 以及一個(gè)基于LLMs的自主復雜任務(wù)規劃 (Complex task planning, CTP) 處理框架AutoPlan. 該框架可以對復雜任務(wù)進(jìn)行自主規劃得到元任務(wù)序列, 并使用遞進(jìn)式ReAct提示 (Progressive ReAct prompting,PRP) 方法對已規劃的元任務(wù)逐步執行. 該框架的有效性通過(guò)在CTPaE上的實(shí)驗及與其他經(jīng)典算法的比較分析得到了驗證. 項目地址:
https://github.com/LDLINGLINGLING/AutoPlan .Abstract: With the advancement of deep learning and natural language processing technologies, large language models (LLMs) have shown significant potential. Despite their power, they still face limitations when dealing with complex tasks, especially when the tasks require integrative planning and the invocation of external tools. In response to this challenge, this paper proposes the first domestic dataset for complex task planning and execution with a military game context, the Chinese complex task planning and execution dataset (CTPaE), and a new framework for autonomous complex task planning (CTP) using LLMs named AutoPlan. The framework is capable of autonomously planning complex tasks to generate a sequence of meta-tasks, and employs a progressive ReAct prompting (PRP) method to gradually execute the planned meta-tasks. The effectiveness of the framework has been validated through experiments on the CTPaE and comparative analysis with other classic algorithms. The link of project:https://github.com/LDLINGLINGLING/AutoPlan .-
Key words:
- Large language models (LLMs) /
- tool-use /
- multi-hop reasoning /
- deep learning
-
圖 1 復雜任務(wù)處理框架AutoPlan示意圖
Fig. 1 Diagram of AutoPlan framework for complex task processing
圖 2 元任務(wù)之間的邏輯關(guān)系示意圖
Fig. 2 Diagram illustrating the logical relationships between meta-tasks
表 1 元任務(wù)的屬性
Table 1 Properties of meta-tasks
任務(wù)屬性 符號表示 屬性描述 所在位置 $ s_{i} $ 在序列中的邏輯關(guān)系 工具需求 $ a_{i} $ 執行該任務(wù)的工具需求 參數配置 $ p_{i} $ 調用工具時(shí)的參數配置 運行結果 $ r_{i} $ 該任務(wù)的運行結果 
下載: 導出CSV
表 2 CTPaE涉及的工具名稱(chēng)和功能介紹
Table 2 The name and function introduction of the tools involved in the CTPaE
工具名稱(chēng) 功能 google_search 通用搜索引擎, 可訪(fǎng)問(wèn)互聯(lián)網(wǎng)、查詢(xún)信息等 military_information_search 軍事搜索引擎, 可訪(fǎng)問(wèn)軍事內部網(wǎng)絡(luò )、查詢(xún)情報等 address_book 獲取如電話(huà)、郵箱、地址等個(gè)人信息 email 發(fā)送和接收郵件 image_gen 根據輸入的文本生成圖像 situation_display 輸入目標位置坐標和顯示范圍、當前敵我雙方的戰場(chǎng)態(tài)勢圖像, 并生成圖片 calendar 獲取當前時(shí)間和日期 map_search 可以查詢(xún)地圖上所有單位位置信息的工具, 返回所有敵軍的位置信息 knowledge_graph 通過(guò)武器裝備知識圖譜獲取各類(lèi)武器裝備的信息 math_formulation 可以通過(guò)Python的eval(·)函數計算出輸入的字符串表達式結果并返回 weapon_launch 武器發(fā)射按鈕是可以啟動(dòng)指定武器打擊指定目標位置的工具 distance_calculation 可以計算給定目標單位之間的距離
下載: 導出CSV
表 3 與相關(guān)方法在CTPaE上的性能比較
Table 3 Performance comparison with related methods on the CTPaE
方法 規模 (B) 評價(jià)指標(%) TSR TCR PT ST ReAct 1.8 7.99 30.30 39.23 34.50 14 37.37 90.00 60.57 48.99 72 39.24 76.40 68.33 60.04 TPTU 1.8 0.60 18.80 33.07 24.92 14 36.13 87.30 60.19 48.30 72 39.84 76.80 68.14 59.96 AutoPlan 1.8 18.70 45.70 91.11 48.15 14 52.30 94.70 90.81 79.24 72 87.02 99.90 99.34 97.09
下載: 導出CSV
表 4 不同任務(wù)規劃方法性能比較
Table 4 Performance comparison of different task planning methods
方法 規模 (B) 評價(jià)指標(%) TSR TCR PT ST 不進(jìn)行規劃 1.8 7.99 30.30 39.23 34.50 14 37.37 90.00 60.57 48.99 72 39.24 76.40 68.33 60.04 TPTU 1.8 0.60 18.80 33.07 24.92 14 36.13 87.30 60.19 48.30 72 39.84 76.80 68.14 59.96 CTP 1.8 7.27 30.20 39.23 34.50 14 37.54 89.90 60.63 49.02 72 39.48 76.90 68.01 59.82 人工規劃 1.8 8.17 43.38 39.24 34.50 14 47.70 92.05 83.54 72.21 72 61.69 97.60 86.78 80.75
下載: 導出CSV
表 5 不同任務(wù)執行策略性能比較
Table 5 Performance comparison of different task execution strategies
任務(wù)規劃方法 任務(wù)執行方法 規模 (B) 評價(jià)指標(%) TSR TCR PT ST 人工規劃 ReAct 1.8 8.17 43.38 39.24 34.50 14 47.70 92.05 83.54 72.21 72 61.69 97.60 86.78 80.75 PRP 1.8 18.39 (+10.22) 45.60 (+2.22) 91.15 (+51.91) 48.28 (+13.78) 14 53.29 (+5.59) 94.70 (+2.65) 91.07 (+7.53) 79.44 (+7.23) 72 86.43 (+24.74) 99.90 (+2.30) 99.47 (+12.69) 97.89 (+17.14) CTP ReAct 1.8 7.27 30.20 39.23 34.50 14 37.54 89.90 60.63 49.02 72 39.48 76.90 68.01 59.82 PRP 1.8 18.70 (+11.43) 45.70 (+15.50) 91.11 (+51.88) 48.15 (+13.65) 14 52.30 (+14.76) 94.70 (+4.80) 90.81 (+30.18) 79.24 (+30.22) 72 87.02 (+47.54) 99.90 (+23.00) 99.34 (+31.33) 97.09 (+37.27)
下載: 導出CSV
亚洲第一网址_国产国产人精品视频69_久久久久精品视频_国产精品第九页<button id="qm3rj"><thead id="qm3rj"></thead></button> <samp id="qm3rj"></samp> <source id="qm3rj"><menu id="qm3rj"><pre id="qm3rj"></pre></menu></source> <video id="qm3rj"><code id="qm3rj"></code></video> <tt id="qm3rj"><track id="qm3rj"></track></tt> -
[1] 盧經(jīng)緯, 郭超, 戴星原, 繆青海, 王興霞, 楊靜, 等. 問(wèn)答ChatGPT之后: 超大預訓練模型的機遇和挑戰. 自動(dòng)化學(xué)報, 2023, 49(4): 705?717Lu Jing-Wei, Guo Chao, Dai Xing-Yuan, Miao Qing-Hai, Wang Xing-Xia, Yang Jing, et al. The ChatGPT after: Opportunities and challenges of very large scale pre-trained models. Acta Automatica Sinica, 2023, 49(4): 705?717 [2] Zhao W X, Zhou K, Li J Y, Tang T Y, Wang X L, Hou Y P, et al. A survey of large language models. arXiv preprint arXiv: 2303.18223, 2023. [3] Dong Q X, Li L, Dai D M, Zheng C, Wu Z Y, Chang B B, et al. A survey for in-context learning. arXiv preprint arXiv: 2301.00234, 2023. [4] Wei J, Wang X Z, Schuurmans D, Bosma M, Ichter B, Xia F, et al. Chain-of-thought prompting elicits reasoning in large language models. arXiv preprint arXiv: 2201.11903, 2023. [5] Cui J X, Li Z J, Yan Y, Chen B H, Yuan L. ChatLaw: Open-source legal large language model with integrated external knowledge bases. arXiv preprint arXiv: 2306.16092, 2023. [6] Nakano R, Hilton J, Balaji S, Wu J, Ouyang L, Kim C, et al. WebGPT: Browser-assisted question-answering with human feedback. arXiv preprint arXiv: 2112.09332, 2022. [7] Schick T, Dwivedi-Yu J, Dessì R, Raileanu R, Lomeli M, Zettlemoyer L, et al. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv: 2302.04761, 2023. [8] Ruan J Q, Chen Y H, Zhang B, Xu Z W, Bao T P, Du G Q, et al. TPTU: Task planning and tool usage of large language model-based AI agents. arXiv preprint arXiv: 2308.03427, 2023. [9] Patel A, Bhattamishra S, Goyal N. Are NLP models really able to solve simple math word problems? arXiv preprint arXiv: 2103.07191, 2021. [10] Wang F Y. New control paradigm for industry 5.0: From big models to foundation control and management. IEEE/CAA Journal of Automatica Sinica, 2023, 10(8): 1643?1646 [11] Wang X X, Yang J, Wang Y T, Miao Q H, Wang F Y, Zhao A J, et al. Steps toward industry 5.0: Building “6S” parallel industries with cyber-physical-social intelligence. IEEE/CAA Journal of Automatica Sinica, 2023, 10(8): 1692?1703 doi: 10.1109/JAS.2023.123753 [12] Song C H, Wu J M, Washington C, Sadler B M, Chao W L, Su Y. LLM-Planner: Few-shot grounded planning for embodied agents with large language models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Vancouver, Canada: IEEE, 2013. 2998?3009 [13] 陽(yáng)東升, 盧經(jīng)緯, 李強, 王飛躍. 超大預訓練模型在指揮控制領(lǐng)域的應用與挑戰. 指揮與控制學(xué)報, 2023, 9(2): 146?155Yang Dong-Sheng, Lu Jing-Wei, Li Qiang, Wang Fei-Yue. Issues and challenges of ChatGPT-like large scale pre-trained model for command and control. Journal of Command and Control, 2023, 9(2): 146?155 [14] Wang F Y. Parallel intelligence in metaverses: Welcome to Hanoi! IEEE Intelligent Systems, 2022, 37(1): 16?20 [15] Gao L Y, Madaan A, Zhou S Y, Alon U, Liu P F, Yang Y M, et al. PAL: Program-aided language models. In: Proceedings of the 40th International Conference on Machine Learning. Hawaii, USA: PMLR, 2023. 10764?10799 [16] Parisi A, Zhao Y, Fiedel N. TALM: Tool augmented language models. arXiv preprint arXiv: 2205.12255, 2022. [17] Qin Y J, Liang S H, Ye Y N, Zhu K L, Yan L, Lu Y X, et al. ToolLLM: Facilitating large language models to master 16000+ real-world APIS. arXiv preprint arXiv: 2307.16789, 2023. [18] 楊靜, 王曉, 王雨桐, 劉忠民, 李小雙, 王飛躍. 平行智能與CPSS: 三十年發(fā)展的回顧與展望. 自動(dòng)化學(xué)報, 2023, 49(3): 614?634Yang Jing, Wang Xiao, Wang Yu-Tong, Liu Zhong-Min, Li Xiao-Shuang, Wang Fei-Yue. Parallel intelligence and CPSS in 30 years: An ACP approach. Acta Automatica Sinica, 2023, 49(3): 614?634 [19] Wu W S, Yang W Y, Li J J, Zhao Y, Zhu Z Q, Chen B, et al. Autonomous crowdsensing: Operating and organizing crowdsensing for sensing automation. IEEE Transactions on Intelligent Vehicles, DOI: 10.1109/TIV.2024.3355508 [20] Wang Y T, Wang X, Wang X X, Yang J, Kwan O, Li L X, et al. The ChatGPT after: Building knowledge factories for knowledge workers with knowledge automation. IEEE/CAA Journal of Automatica Sinica, 2023, 10(11): 2041?2044 doi: 10.1109/JAS.2023.123966 [21] Paranjape B, Lundberg S, Singh S, Hajishirzi H, Zettlemoyer L, Ribeiro M T. ART: Automatic multi-step reasoning and tool-use for large language models. arXiv preprint arXiv: 2303.09014, 2023. [22] Wang B, Li G, Li Y. Enabling conversational interaction with mobile UI using large language models. In: Proceedings of the CHI Conference on Human Factors in Computing Systems. Hamburg, Germany: ACM, 2023. 1?17 [23] Li H X, Su J R, Chen Y T, Li Q, Zhang Z X. SheetCopilot: Bringing software productivity to the next level through large language models. arXiv preprint arXiv: 2305.19308, 2023. [24] Chen Z P, Zhou K, Zhang B C, Gong Z, Zhao W X, Wen J R. ChatCoT: Tool-augmented chain-of-thought reasoning on chat-based large language models. arXiv preprint arXiv: 2305.14323, 2023. [25] Patil S G, Zhang T J, Wang X, Gonzalez J E. Gorilla: Large language model connected with massive APIs. arXiv preprint arXiv: 2305.15334, 2023. [26] Hao S B, Liu T Y, Wang Z, Hu Z T. ToolkenGPT: Augmenting frozen language models with massive tools via tool embeddings. arXiv preprint arXiv: 2305.11554, 2024. [27] Shen Y L, Song K T, Tan X, Li D S, Lu W M, Zhuang Y T, et al. HuggingGPT: Solving AI tasks with ChatGPT and its friends in hugging face. arXiv preprint arXiv: 2303.17580, 2023. [28] Bai J Z, Bai S, Chu Y F, Cui Z Y, Dang K, Deng X D, et al. QWEN technical report. arXiv preprint arXiv: 2309.16609, 2023. [29] Yao S Y, Zhao J, Yu D, Du N, Shafran I, Narasimhan K, et al. ReAct: Synergizing reasoning and acting in language models. arXiv preprint arXiv: 2210.03629, 2023. [30] Wang Y Z, Kordi Y, Mishra S, Liu A, Smith N A, Khashabi D, et al. Self-instruct: Aligning language models with self-generated instructions. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. Toronto, Canada: ACL, 2023. 13484?13508 [31] Qin Y J, Cai Z H, Jin D, Yan L, Liang S H, Zhu K L, et al. WebCPM: Interactive web search for Chinese long-form question answering. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. Toronto, Canada: ACL, 2023. 8968?8988 [32] Kim G, Baldi P, McAleer S. Language models can solve computer tasks. arXiv preprint arXiv: 2303.17491, 2023. [33] Wu W S, Chang T, Li X M, Yin Q J, Hu Y. Vision-language navigation: A survey and taxonomy. Neural Computing and Applications, 2024, 36(7): 3291?3316 doi: 10.1007/s00521-023-09217-1 [34] Vemprala S, Bonatti R, Bucker A, Kapoor A. ChatGPT for robotics: Design principles and model abilities. arXiv preprint arXiv: 2306.17582, 2023. [35] Driess D, Xia F, Sajjadi M, Lynch C, Chowdhery A, Ichter B, et al. PaLM-E: An embodied multimodal language model. arXiv preprint arXiv: 2303.03378, 2023. [36] Du Z X, Qian Y J, Liu X, Ding M, Qiu J Z, Yang Z L, et al. GLM: General language model pretraining with autoregressive blank infilling. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Dublin, Ireland: ACL, 2023. 320–335 [37] Xiao G X, Tian Y D, Chen B D, Han S, Lewis M. Efficient streaming language models with attention sinks. arXiv preprint arXiv: 2309.17453, 2023. -
圖(5) / 表(5)
計量
- 文章訪(fǎng)問(wèn)數: 1305
- HTML全文瀏覽量: 923
- PDF下載量: 501
- 被引次數: 0
