不確定性環(huán)境下維納模型的隨機變分貝葉斯學(xué)習

劉切; 李俊豪; 王浩; 曾建學(xué); 柴毅

doi:10.16383/j.aas.c210925

不確定性環(huán)境下維納模型的隨機變分貝葉斯學(xué)習

doi: 10.16383/j.aas.c210925

1.
重慶大學(xué)自動(dòng)化學(xué)院重慶 400044

基金項目: 國家重點(diǎn)研發(fā)計劃(2021YFB1715000), 國家自然科學(xué)基金(61903051, U2034209)資助

詳細信息

作者簡(jiǎn)介:
劉切：重慶大學(xué)自動(dòng)化學(xué)院副教授. 2016年獲得北京化工大學(xué)控制科學(xué)與工程專(zhuān)業(yè)博士學(xué)位. 主要研究方向為人工智能及其在復雜過(guò)程的控制和優(yōu)化中的應用. 本文通信作者. E-mail: qieliu@cqu.edu.cn

李俊豪：重慶大學(xué)自動(dòng)化學(xué)院碩士研究生. 2020年獲得西安理工大學(xué)自動(dòng)化與信息工程學(xué)院學(xué)士學(xué)位. 主要研究方向為系統辨識與人工智能. E-mail: 202013021042@cqu.edu.cn

王浩：重慶大學(xué)自動(dòng)化學(xué)院碩士研究生. 2021年獲得安徽師范大學(xué)物理與電子信息學(xué)院學(xué)士學(xué)位. 主要研究方向為模型預測與人工智能. E-mail: 202113021007@cqu.edu.cn

曾建學(xué)：重慶大學(xué)自動(dòng)化學(xué)院碩士研究生. 2018年獲得華北科技學(xué)院電子信息工程學(xué)院學(xué)士學(xué)位. 主要研究方向為容器技術(shù)與人工智能. E-mail: lc9zjx@126.com

柴毅：重慶大學(xué)自動(dòng)化學(xué)院教授. 2001年獲得重慶大學(xué)博士學(xué)位. 主要研究方向為信息融合, 故障診斷, 智能控制系統. E-mail: chaiyi@cqu.edu.cn

計量
- 文章訪(fǎng)問(wèn)數: 765
- HTML全文瀏覽量: 250
- PDF下載量: 131
- 被引次數: 0
出版歷程
- 收稿日期: 2021-09-27
- 錄用日期: 2022-03-01
- 網(wǎng)絡(luò )出版日期: 2022-05-08
- 刊出日期: 2024-06-27

Stochastic Variational Bayesian Learning of Wiener Model in the Presence of Uncertainty

LIU Qie^1
,,
LI Jun-Hao^1
,,
WANG Hao^1
,,
ZENG Jian-Xue^1
,,
CHAI Yi^1
,

1.
School of Automation, Chongqing University, Chongqing 400044

Funds: Supported by National Key Research and Development Program of China (2021YFB1715000) and National Natural Science Foundation of China (61903051, U2034209)

More Information

Author Bio:
LIU Qie　Associate professor at the School of Automation, Chongqing University. He received his Ph.D. degree from Beijing University of Chemical Technology in 2016. His research interest covers artificial intelligence and its applications on the control and optimization of complex processes. Corresponding author of this paper

LI Jun-Hao　Master student at the School of Automation, Chongqing University. He received his bachelor degree from the School of Automation and Information Engineering, Xi＇an University of Technology in 2020. His research interest covers system identification and artificial intelligence

WANG Hao　Master student at the School of Automation, Chongqing University. He received his bachelor degree from the School of Physics and Electronic Information, Anhui Normal University in 2021. His research interest covers model prediction and artificial intelligence

ZENG Jian-Xue　Master student at the School of Automation, Chongqing University. He received his bachelor degree from the School of Electronic Information Engineering, North China Institute of Science and Technology in 2018. His research interest covers container and artificial intelligence

CHAI Yi　Professor at the School of Automation, Chongqing University. He received his Ph.D. degree from Chongqing University in 2001. His research interest covers information fusion, fault diagnosis, and intelligent control system

摘要

摘要: 多重不確定性環(huán)境下的非線(xiàn)性系統辨識是一個(gè)開(kāi)放問(wèn)題. 貝葉斯學(xué)習在描述、處理不確定性方面具有顯著(zhù)優(yōu)勢, 已在線(xiàn)性系統辨識方面得到廣泛應用, 但在非線(xiàn)性系統辨識的應用較少, 且面臨概率估計復雜、計算量大等難題. 針對上述問(wèn)題, 以典型維納(Wiener)非線(xiàn)性過(guò)程為對象, 提出基于隨機變分貝葉斯的非線(xiàn)性系統辨識方法. 首先對過(guò)程噪聲、測量噪聲以及參數不確定性進(jìn)行概率描述; 然后利用隨機變分貝葉斯方法對模型參數進(jìn)行后驗估計. 在估計過(guò)程中, 利用隨機優(yōu)化思想, 僅利用部分中間變量概率信息估計模型參數分布的自然梯度期望, 與利用所有中間變量概率信息估計模型參數比較, 顯著(zhù)降低了計算復雜性. 該方法是首次在系統辨識領(lǐng)域中的應用. 最后, 利用一個(gè)仿真實(shí)例和一個(gè)維納模型的Benchmark問(wèn)題, 證明了該方法在對大規模數據下非線(xiàn)性系統辨識的有效性.
- 非線(xiàn)性系統辨識 /
- 隨機優(yōu)化 /
- 變分貝葉斯 /
- 維納模型
Abstract: Nonlinear system identification in multiple uncertain environment is an open problem. Bayesian learning has significant advantages in describing and dealing with uncertainties and has been widely used in linear system identification. However, the use of Bayesian learning for nonlinear system identification has not been well studied, confronted with the complexity of the estimation of the probability and the high computational cost. Motivated by these problems, this paper proposes a nonlinear system identification method based on stochastic variational Bayesian for Wiener model, a typical nonlinear model. First, the process noise, measurement noise and parameter uncertainty are described in terms of probability distribution. Then, the posterior estimation of model parameters is carried out by using the stochastic variational Bayesian approach. In this framework, only a few intermediate variables are used to estimate the natural gradient of the lower bound function of the likelihood function based on the stochastic optimization idea. Compared with classical variational Bayesian approach, where the estimation of model parameters depends on the information of all the intermediate variables, the computational complexity is significantly reduced for the proposed method since it only depends on the information of a few intermediate variables. To the best of our knowledge, it is the first time to use the stochastic variational Bayesian to system identification. A numerical example and a Benchmark problem of Wiener model are used to show the effectiveness of this method in the nonlinear system identification in the presence of large-scale data.
- Nonlinear system identification /
- stochastic optimization /
- variational Bayesian /
- Wiener model

HTML全文

圖 1 維納模型結構示意圖

Fig. 1 The structure of Wiener model

下載: 全尺寸圖片幻燈片

圖 2 SVBI中目標函數的更新示意圖

Fig. 2 The update process of the objective function in SVBI

下載: 全尺寸圖片幻燈片

圖 3 辨識參數的收斂狀況

Fig. 3 Convergence of identified parameters

下載: 全尺寸圖片幻燈片

圖 4 下界函數的收斂過(guò)程

Fig. 4 Convergence process of the lower bound function

下載: 全尺寸圖片幻燈片

圖 5 預測輸出與實(shí)際輸出比較

Fig. 5 Comparison of predicted output with actual output

下載: 全尺寸圖片幻燈片

圖 6 系統預測輸出與實(shí)際輸出

Fig. 6 Predicted output and actual output of the system

下載: 全尺寸圖片幻燈片

表 1 不同子采樣數據點(diǎn)對應的參數辨識情況

Table 1 Identification of parameters corresponding to different sub-sampling data points

	$ \langle \theta _0 \rangle $	$ \langle \theta _1 \rangle $	$ \langle \theta _2 \rangle $	$ \langle \theta _3 \rangle $	$ \langle \theta _4 \rangle $	$ \langle \lambda _0 \rangle $	$ \langle \lambda _1 \rangle $	$ \langle \lambda _2 \rangle $	時(shí)間(s)
真實(shí)值	1	?0.5000	0.2500	?0.1250	0.0625	0	1	1	—
采樣1個(gè)點(diǎn)	1±0	?0.5463±0.3604	0.2507±0.2471	?0.2446±0.2655	0.0358±0.2882	0.5434±0.4180	0.6625±0.2907	0.3803±0.2185	0.6005
采樣5%	1±0	?0.5060±0.0330	0.2693±0.0497	?0.1252±0.0323	0.0633±0.0323	0.0908±0.2707	0.9871±0.1480	0.9103±0.1246	3.1829
采樣10%	1±0	?0.5055±0.0248	0.2571±0.0257	?0.1341±0.0255	0.0594±0.0256	0.0631±0.0504	0.9684±0.0498	0.9499±0.0459	7.7402
采樣20%	1±0	?0.5077±0.0204	0.2544±0.0202	?0.1287±0.0289	0.0659±0.0291	0.0575±0.0540	0.9813±0.0518	0.9574±0.0451	11.4620
采樣全部	1±0	?0.5078±0.0278	0.2541±0.0283	?0.1299±0.0271	0.0685±0.0246	0.0777±0.0726	0.9439±0.1183	0.9252±0.1326	9.0772

下載: 導出CSV

表 2 不同異常值存在時(shí)的參數辨識情況

Table 2 Parameter identification when different outliers exist

	$ \langle \theta _0 \rangle $	$ \langle \theta _1 \rangle $	$ \langle \theta _2 \rangle $	$ \langle \theta _3 \rangle $	$ \langle \theta _4 \rangle $	$ \langle \theta _5 \rangle $	時(shí)間 (s)
真實(shí)值	1	?0.5000	0.2500	?0.1250	0.0625	?0.03125	—
無(wú)異常值	1±0	?0.4989±0.0292	0.2495±0.0293	?0.1254±0.0223	0.0611±0.0257	?0.0338±0.0262	2.9369
2% 異常值	1±0	?0.5097±0.0389	0.2672±0.0497	?0.1305±0.0426	0.0652±0.0452	?0.0291±0.0494	2.9480
5% 異常值	1±0	?0.5060±0.0330	0.2693±0.0497	?0.1252±0.0323	0.0633±0.0323	?0.0314±0.0523	3.1829
10% 異常值	1±0	?0.5349±0.0325	0.2627±0.0323	?0.1314±0.0330	0.0685±0.0389	?0.0377±0.0355	2.9057

下載: 導出CSV

表 3 不同辨識方法的性能比較

Table 3 Performance comparison of different recognition methods

		$ b_0 $	$ a_1 $	$ \langle \lambda _0 \rangle(\lambda_0) $	$ \langle \lambda _1 \rangle(\lambda_1) $	$ \langle \lambda _2 \rangle(\lambda_2) $	均方誤差	時(shí)間(s)
真實(shí)值		1	0.5	0	1	1	—	—
無(wú)異常值	SVBI	—	—	0.0648±0.0620	0.9633±0.0509	0.9766±0.0626	0.9136	2.936 9
	VBEM	—	—	0.0503±0.0346	0.9411±0.0393	0.9655±0.0459	0.8978	9.7046
	MLE	1±0	0.5102±0.0136	0.1054±0.0405	1.0154±0.0464	0.9490±0.0411	0.9130	9.0350
	PEM	1±0	0.4948±0.0172	0.0828±0.0524	0.9905±0.0373	1.0072±0.0449	0.9132	0.6474
5% 異常值	SVBI	—	—	0.0575±0.0540	0.9813±0.0520	0.9573±0.0450	5.4540	2.9352
	VBEM	—	—	0.0503±0.0411	0.9770±0.0532	0.9748±0.0518	3.8695	9.7709
	MLE	1±0	0.4150±0.0711	?0.9407±0.1253	1.0019±0.1839	1.3715±0.1895	3.9574	9.6693
	PEM	1±0	0.4999±0.0549	0.1072±0.1871	0.9646±0.1926	0.9878±0.1558	3.8374	0.6580
10% 異常值	SVBI	—	—	0.1439±0.1065	0.9163±0.0924	0.8416±0.0924	7.5364	2.9057
	VBEM	—	—	0.0556±0.0468	0.9711±0.0538	0.9568±0.0553	5.5110	9.9245
	MLE	—	—	—	—	—	—	—
	PEM	1±0	0.4723±0.2004	0.1458±0.5211	0.9746±0.3091	1.0030±0.3253	5.4992	0.6620

下載: 導出CSV

表 4 式(52)部分參數辨識結果

Table 4 The identification results of the part parameters of the process (52)

參數	$\theta_0$	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	$\theta_5$	$\theta_6$	$\theta_7$	$\theta_8$	$\theta_9$	$c_0$	$c_1$	$c_2$	$Q$	$R$
結果值	?0.0390	0.0648	?0.0547	0.0856	?0.0462	0.2613	0.0501	0.2041	0.3396	0.4154	?0.0188	0.1035	?0.0030	0.0034	0.0014

下載: 導出CSV

表 5 不同方法的性能比較

Table 5 Performance comparison of different methods

采樣點(diǎn)數	方法	均方誤差(V)	參數個(gè)數	時(shí)間(s)
2 000	SVBI	0.056 95	25	256.12
	VBEM	0.062 83	25	1 211.27
	SVBI	0.034 07	40	264.27
	VBEM	0.034 25	40	1 214.55
10 000	SVBI	0.061 79	25	1 299.99
	VBEM	0.093 34	25	6 347.28
	SVBI	0.033 85	40	1 332.31
	VBEM	0.034 04	40	6 442.98

下載: 導出CSV

<button id="qm3rj"><thead id="qm3rj"></thead></button>

亚洲第一网址_国产国产人精品视频69_久久久久精品视频_国产精品第九页

參考文獻(28)

[1]	王樂(lè )一, 趙文虓. 系統辨識: 新的模式、挑戰及機遇. 自動(dòng)化學(xué)報, 2013, 39(7): 933?942 doi: 10.1016/S1874-1029(13)60062-2 Wang Le-Yi, Zhao Wen-Xiao. System identification: New paradigms, challenges, and opportunities. Acta Automatica Sinica, 2013, 39(7): 933?942 doi: 10.1016/S1874-1029(13)60062-2
[2]	劉鑫. 時(shí)滯取值概率未知下的線(xiàn)性時(shí)滯系統辨識方法. 自動(dòng)化學(xué)報, 2023, 49(10): 2136?2144 Liu Xin. Identification of linear time-delay systems with unknown delay distributions in its value range. Acta Automatica Sinica, 2023, 49(10): 2136?2144
[3]	Stoica P. On the convergence of an iterative algorithm used for Hammerstein system identification. IEEE Transactions on Automatic Control, 1981, 26(4): 967?969 doi: 10.1109/TAC.1981.1102761
[4]	張亞軍, 柴天佑, 楊杰. 一類(lèi)非線(xiàn)性離散時(shí)間動(dòng)態(tài)系統的交替辨識算法及應用. 自動(dòng)化學(xué)報, 2017, 43(1): 101?113 Zhang Ya-Jun, Chai Tian-You, Yang Jie. Alternating identification algorithm and its application to a class of nonlinear discrete-time dynamical systems. Acta Automatica Sinica, 2017, 43(1): 101?113
[5]	黃玉龍, 張勇剛, 李寧, 趙琳. 一種帶有色量測噪聲的非線(xiàn)性系統辨識方法. 自動(dòng)化學(xué)報, 2015, 41(11): 1877?1892 Huang Yu-Long, Zhang Yong-Gang, Li Ning, Zhao Lin. An identification method for nonlinear systems with colored measurement noise. Acta Automatica Sinica, 2015, 41(11): 1877?1892
[6]	Ljung L. Perspectives on system identification. Annual Reviews in Control, 2008, 34(1): 1?12
[7]	Sch?n T B, Wills A, Ninness B. System identification of nonlinear state-space models. Automatica, 2011, 47(1): 39?49 doi: 10.1016/j.automatica.2010.10.013
[8]	Billings S A. Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains. Chichester: John Wiley & Sons, 2013.
[9]	Carini A, Orcioni S, Terenzi A, Cecchi S. Nonlinear system identification using Wiener basis functions and multiple-variance perfect sequences. Signal Processing, 2019, 160: 137?149 doi: 10.1016/j.sigpro.2019.02.017
[10]	Schoukens M, Tiels K. Identification of block-oriented nonlinear systems starting from linear approximations: A survey. Automatica, 2017, 85: 272?292 doi: 10.1016/j.automatica.2017.06.044
[11]	Bershad N J, Celka P, McLaughlin S. Analysis of stochastic gradient identification of Wiener-Hammerstein systems for nonlinearities with Hermite polynomial expansions. IEEE Transactions on Signal Processing, 2001, 49(5): 1060?1072 doi: 10.1109/78.917809
[12]	Valarmathi K, Devaraj D, Radhakrishnan T K. Intelligent techniques for system identification and controller tuning in pH process. Brazilian Journal of Chemical Engineering, 2009, 26(1): 99?111 doi: 10.1590/S0104-66322009000100010
[13]	Zhu Y. Distillation column identification for control using Wiener model. In: Proceedings of the American Control Conference. San Diego, USA: IEEE, 1999. 3462?3466
[14]	Schoukens J, Suykens J, Ljung L. Wiener-Hammerstein benchmark. In: Proceedings of the 15th IFAC Symposium on System Identification (SYSID 2009). Malo, France: 2009.
[15]	Hagenblad A, Ljung L, Wills A. Maximum likelihood identification of Wiener models. Automatica, 2008, 44(11): 2697?2705 doi: 10.1016/j.automatica.2008.02.016
[16]	Xu W Y, Bai E W, Cho M. System identification in the presence of outliers and random noises: A compressed sensing approach. Automatica, 2014, 50(11): 2905?2911 doi: 10.1016/j.automatica.2014.10.017
[17]	Bottegal G, Castro-Garcia R, Suykens J A K. A two-experiment approach to Wiener system identification. Automatica, 2018, 93: 282?289 doi: 10.1016/j.automatica.2018.03.069
[18]	Westwick D T, Schoukens J. Initial estimates of the linear subsystems of Wiener-Hammerstein models. Automatica, 2012, 48(11): 2931?2936 doi: 10.1016/j.automatica.2012.06.091
[19]	Giordano G, Gros S, Sj?berg J. An improved method for Wiener-Hammerstein system identification based on the fractional approach. Automatica, 2018, 94: 349?360 doi: 10.1016/j.automatica.2018.04.046
[20]	Liu Q, Lin W Y, Jiang S L, Chai Y, Sun L. Robust estimation of Wiener models in the presence of outliers using the VB approach. IEEE Transactions on Industrial Electronics, 2021, 68(11): 11390?11399 doi: 10.1109/TIE.2020.3028806
[21]	Xie L, Yang H Z, Huang B. FIR model identification of multirate processes with random delays using EM algorithm. AIChE Journal, 2013, 59(11): 4124?4132 doi: 10.1002/aic.14147
[22]	Agamennoni G, Nieto J I, Nebot E M. Approximate inference in state-space models with heavy-tailed noise. IEEE Transactions on Signal Processing, 2012, 60(10): 5024?5037 doi: 10.1109/TSP.2012.2208106
[23]	Bishop C M. Pattern Recognition and Machine Learning. New York: Springer, 2006.
[24]	Amari S I. Natural gradient works efficiently in learning. Neural Computation, 1998, 10(2): 251?276 doi: 10.1162/089976698300017746
[25]	Amari S I. Differential geometry of curved exponential families-curvatures and information loss. The Annals of Statistics, 1982, 10(2): 357?385
[26]	Bottou L. On-line learning and stochastic approximations. On-line Learning in Neural Networks. New York: Cambridge University Press, 1999.
[27]	Robbins H, Monro S. A stochastic approximation method. The Annals of Mathematical Statistics, 1951, 22(3): 400?407 doi: 10.1214/aoms/1177729586
[28]	Hoffman M D, Blei D M, Wang C, Paisley J. Stochastic variational inference. The Journal of Machine Learning Research, 2013, 14(1): 1303?1347