不確定性環境下維納模型的隨機變分貝葉斯學習

劉切; 李俊豪; 王浩; 曾建學; 柴毅

doi:10.16383/j.aas.c210925

不確定性環境下維納模型的隨機變分貝葉斯學習

doi: 10.16383/j.aas.c210925

劉切^1,,
李俊豪^1,,
王浩^1,,
曾建學^1,,
柴毅^1,

1.
重慶大學自動化學院重慶 400044

基金項目: 國家重點研發計劃(2021YFB1715000), 國家自然科學基金(61903051, U2034209)資助

詳細信息

作者簡介:
劉切：重慶大學自動化學院副教授. 2016年獲得北京化工大學控制科學與工程專業博士學位. 主要研究方向為人工智能及其在復雜過程的控制和優化中的應用. 本文通信作者. E-mail: qieliu@cqu.edu.cn

李俊豪：重慶大學自動化與信息工程學院碩士研究生. 2020年獲得西安理工大學自動化學院學士學位. 主要研究方向為系統辨識與人工智能. E-mail: 202013021042@cqu.edu.cn

王浩：重慶大學自動化學院碩士研究生. 2021年獲得安徽師范大學物理與電子信息學院學士學位. 主要研究方向為模型預測與人工智能. E-mail: 202113021007@cqu.edu.cn

曾建學：重慶大學自動化學院碩士研究生. 2018年獲得華北科技學院電子信息工程學院學士學位. 主要研究方向為容器技術與人工智能. E-mail: lc9zjx@126.com

柴毅：重慶大學自動化學院教授. 2001年獲得重慶大學博士學位. 主要研究方向為信息融合, 故障診斷, 智能控制系統. E-mail: chaiyi@cqu.edu.cn

計量
- 文章訪問數: 526
- HTML全文瀏覽量: 172
- 被引次數: 0
出版歷程
- 收稿日期: 2021-09-27
- 錄用日期: 2022-03-01
- 網絡出版日期: 2022-05-08

Stochastic Variational Bayesian Learning of Wiener Model in the Presence of Uncertainty

LIU Qie^1
,,
LI Jun-Hao^1
,,
WANG Hao^1
,,
ZENG Jian-Xue^1
,,
CHAI Yi^1
,

1.
School of Automation, Chongqing University, Chongqing 400044

Funds: Supported by National Key Research and Development Program of China (2021YFB1715000), National Natural Science Foundation of China (61903051, U2034209)

More Information

Author Bio:
LIU Qie　Associate professor at the School of Automation, Chongqing University. He received his Ph.D. degree from the Beijing University of Chemical Technology in 2016. His research interest covers artificial intelligence and its applications on the control and optimization of complex processes. Corresponding author of this paper

LI Jun-Hao　Master student at the School of Automation, Chongqing University. He received his bachelor degree from Xi＇an University of Technology in 2020. His research interest covers system identification and artificial intelligence

WANG Hao　Master student at the School of Automation, Chongqing University. He received his bachelor degree from Anhui Normal University of Physics and Electronic in 2021. His research interest covers model prediction and artificial intelligence

ZENG Jian-Xue　Master student at the School of Automation, Chongqing University. He received his bachelor degree from North China Institute Of Science And Technology in 2018. His research interests covers container and artificial intelligence

CHAI Yi　Professor at the School of Automation, Chongqing University. He received his Ph.D. degree from Chongqing University in 2001. His research interest covers information fusion, fault diagnosis, and intelligent control system

摘要

摘要: 多重不確定性環境下的非線性系統辨識是一個開放問題. 貝葉斯學習在描述、處理不確定性方面具有顯著優勢, 已在線性系統辨識方面得到廣泛應用, 但在非線性系統辨識的應用較少, 面臨概率估計復雜、計算量大等困難. 針對上述問題, 以典型維納(Wiener)非線性過程為對象, 提出基于隨機變分貝葉斯的非線性系統辨識方法. 首先對過程噪聲、測量噪聲以及參數不確定性進行概率描述; 然后利用隨機變分貝葉斯方法對模型參數進行后驗估計. 在估計過程中, 利用隨機優化思想, 僅利用部分中間變量概率信息估計模型參數分布的自然梯度期望, 與利用所有中間變量概率信息估計模型參數比較, 顯著降低了計算復雜性. 該方法是首次在系統辨識領域中的應用. 本文利用一個仿真實例和一個維納模型的Benchmark問題, 證明了該方法在對大規模數據系統辨識時的有效性.
- 非線性系統辨識 /
- 隨機優化 /
- 變分貝葉斯 /
- 維納模型
Abstract: Nonlinear system identification in uncertain environment is an open problem. Bayesian learning has significant advantages in describing and dealing with uncertainties and has been widely used in linear system identification. However, the use of Bayesian learning for nonlinear system identification has not been well studied due to the complexity of the estimation of the probability, especially in the presence of large-scale data. Motivated by these problems, this paper proposes a nonlinear system identification method based on stochastic variational Bayesian for Wiener model, a typical nonlinear model. First, the process noise, measurement noise and parameter uncertainty are described in terms of probability distribution. Then, the posterior estimation of model parameters is carried out by using the stochastic variational Bayesian approach. In this framework, only a few intermediate variables are used to estimate the natural gradient of the lower bound function of the likelihood function. Compared with classical variational Bayesian approach, where the estimation of model parameters depends on the information of all the intermediate variables, the computational complexity is significantly reduced for the proposed method since it only depends on the information of a few intermediate variables. To the best of our knowledge, it is the first time to use the stochastic variational Bayesian to system identification. A numerical example and a Benchmark problem of Wiener model are used to show the effectiveness of this method in the nonlinear system identification in the presence of large-scale data.
- Nonlinear system identification /
- stochastic optimization /
- variational Bayesian /
- Wiener model

HTML全文

圖 1 維納模型結構示意圖

Fig. 1 The structure of Wiener models

下載: 全尺寸圖片幻燈片

圖 2 SVBI中目標函數的更新示意圖

Fig. 2 The update process of the objective function in SVBI

下載: 全尺寸圖片幻燈片

圖 3 參數的收斂狀況

Fig. 3 Convergence of identified parameters

下載: 全尺寸圖片幻燈片

圖 4 下界函數的收斂過程

Fig. 4 Convergence process of the lower bound function

下載: 全尺寸圖片幻燈片

圖 5 預測輸出與實際輸出

Fig. 5 Comparison of predicted output with actual output

下載: 全尺寸圖片幻燈片

圖 6 系統預測輸出與實際輸出

Fig. 6 Predicted output and actual output of the system

下載: 全尺寸圖片幻燈片

表 1 不同子采樣數據點對應的參數辨識情況

Table 1 Identification of parameters corresponding to different sub-sampling data points

	$ \langle \theta _0 \rangle $	$ \langle \theta _1 \rangle $	$ \langle \theta _2 \rangle $	$ \langle \theta _3 \rangle $	$ \langle \theta _4 \rangle $	$ \langle \lambda _0 \rangle $	$ \langle \lambda _1 \rangle $	$ \langle \lambda _2 \rangle $	時間(s)
真實值	1	?0.5000	0.2500	?0.1250	0.0625	0	1	1	—
采樣1個點	1±0	?0.5463±0.3604	0.2507±0.2471	?0.2446±0.2655	0.0358±0.2882	0.5434±0.4180	0.6625±0.2907	0.3803±0.2185	0.6005
采樣5%	1±0	?0.5060±0.0330	0.2693±0.0497	?0.1252±0.0323	0.0633±0.0323	0.0908±0.2707	0.9871±0.1480	0.9103±0.1246	3.1829
采樣10%	1±0	?0.5055±0.0248	0.2571±0.02567	?0.1341±0.0255	0.0594±0.0256	0.0631±0.0504	0.9684±0.0498	0.9499±0.0459	7.7402
采樣20%	1±0	?0.5077±0.0204	0.2544±0.0202	?0.1287±0.0289	0.0659±0.0291	0.0575±0.0540	0.9813±0.0518	0.9574±0.0451	11.4620
采樣全部	1±0	?0.5078±0.0278	0.2541±0.0283	?0.1299±0.0271	0.0685±0.0246	0.0777±0.0726	0.9439±0.1183	0.9252±0.1326	9.0772

下載: 導出CSV

表 2 不同異常值存在時的參數辨識情況

Table 2 Parameter identification when different outliers exist

	$ \langle \theta _0 \rangle $	$ \langle \theta _1 \rangle $	$ \langle \theta _2 \rangle $	$ \langle \theta _3 \rangle $	$ \langle \theta _4 \rangle $	$ \langle \theta _5 \rangle $	時間 (s)
真實值	1	?0.5000	0.2500	?0.1250	0.0625	?0.03125	—
無異常值	1±0	?0.4989±0.0292	0.2495±0.0293	?0.1254±0.0223	0.0611±0.0257	?0.0338±0.0262	2.9369
2% 異常值	1±0	?0.5097±0.0389	0.2672±0.0497	?0.1305±0.0426	0.0652±0.0452	?0.0291±0.0494	2.9480
5% 異常值	1±0	?0.5060±0.0330	0.2693±0.0497	?0.1252±0.0323	0.0633±0.0323	?0.0314±0.0523	3.1829
10% 異常值	1±0	?0.5349±0.0325	0.2627±0.0323	?0.1314±0.0330	0.0685±0.0389	?0.0377±0.0355	2.9057

下載: 導出CSV

表 3 不同辨識方法的性能比較

Table 3 Performance comparison of different recognition methods

		$ b_0 $	$ a_1 $	$ \langle \lambda _0 \rangle(\lambda_0) $	$ \langle \lambda _1 \rangle(\lambda_1) $	$ \langle \lambda _2 \rangle(\lambda_2) $	均方誤差	時間(s)
真實值		1	0.5	0	1	1	—	—
無異常值	SVBI	—	—	0.0648±0.0620	0.9633±0.0509	0.9766±0.0626	0.9136	2.936 9
	VBEM	—	—	0.0503±0.0346	0.9411±0.0393	0.9655±0.0459	0.8978	9.7046
	MLE	1±0	0.5102±0.0136	0.1054±0.0405	1.0154±0.0464	0.9490±0.0411	0.9130	9.0350
	PEM	1±0	0.4948±0.0172	0.0828±0.0524	0.9905±0.0373	1.0072±0.0449	0.9132	0.6474
5% 異常值	SVBI	—	—	0.0575±0.0540	0.9813±0.0520	0.9573±0.0450	5.4540	2.9352
	VBEM	—	—	0.0503±0.0411	0.9770±0.0532	0.9748±0.0518	3.8695	9.7709
	MLE	1±0	0.4150±0.0711	?0.9407±0.1253	1.0019±0.1839	1.3715±0.1895	3.9574	9.6693
	PEM	1±0	0.4999±0.0549	0.1072±0.1871	0.9646±0.1926	0.9878±0.1558	3.8374	0.6580
10% 異常值	SVBI	—	—	0.1439±0.1065	0.9163±0.0924	0.8416±0.0924	7.5364	2.9057
	VBEM	—	—	0.0556±0.0468	0.9711±0.0538	0.9568±0.0553	5.5110	9.9245
	MLE	—	—	—	—	—	—	—
	PEM	1±0	0.4723±0.2004	0.1458±0.5211	0.9746±0.3091	1.0030±0.3253	5.4992	0.6620

下載: 導出CSV

表 4 過程(52)部分參數辨識結果

Table 4 The identification result of the part of the process (52)

參數	$\theta_0$	$\theta_1$	$\theta_2$	$\theta_3$	$\theta_4$	$\theta_5$	$\theta_6$	$\theta_7$	$\theta_8$	$\theta_9$	$c_0$	$c_1$	$c_2$	$Q$	$R$
結果值	?0.0390	0.0648	?0.0547	0.0856	?0.0462	0.2613	0.0501	0.2041	0.3396	0.4154	?0.0188	0.1035	?0.003	0.0034	0.0014

下載: 導出CSV

表 5 不同方法的性能比較

Table 5 Performance comparison of different methods

采樣點數	方法	均方誤差(V)	參數個數	時間(s)
2 000	SVBI	0.056 95	25	256.12
	VBEM	0.062 83	25	1 211.27
	SVBI	0.034 07	40	264.273
	VBEM	0.034 25	40	1 214.55
10 000	SVBI	0.061 79	25	1 299.99
	VBEM	0.093 34	25	6 347.28
	SVBI	0.033 85	40	1 332.31
	VBEM	0.034 04	40	6 442.98

下載: 導出CSV

<button id="qm3rj"><thead id="qm3rj"></thead></button>

亚洲第一网址_国产国产人精品视频69_久久久久精品视频_国产精品第九页

參考文獻(28)

[1]	王樂一, 趙文虓. 系統辨識: 新的模式、挑戰及機遇. 自動化學報, 2013, 39(7): 933?942 doi: 10.1016/S1874-1029(13)60062-2 Wang Le-Yi, Zhao Wen-Xiao. System identification: New paradigms, challenges, and opportunities. Acta Automatica Sinica, 2013, 39(7): 933?942 doi: 10.1016/S1874-1029(13)60062-2
[2]	劉鑫. 時滯取值概率未知下的線性時滯系統辨識方法. 自動化學報, 2023, 49(10): 2136?2144 Liu Xin. Identification of linear time-delay systems with unknown delay distributions in its value range. Acta Automatica Sinica, 2023, 49(10): 2136?2144
[3]	Stoica P. On the convergence of an iterative algorithm used for Hammerstein system identification. IEEE Transactions on Automatic Control, 1981, 26(4): 967?969 doi: 10.1109/TAC.1981.1102761
[4]	張亞軍, 柴天佑, 楊杰. 一類非線性離散時間動態系統的交替辨識算法及應用. 自動化學報, 2017, 43(1): 101?113 Zhang Ya-Jun, Chai Tian-You, Yang Jie. Alternating identification algorithm and its application to a class of nonlinear discrete-time dynamical systems. Acta Automatica Sinica, 2017, 43(1): 101?113
[5]	黃玉龍, 張勇剛, 李寧, 趙琳. 一種帶有色量測噪聲的非線性系統辨識方法. 自動化學報, 2015, 41(11): 1877?1892 Huang Yu-Long, Zhang Yong-Gang, Li Ning, Zhao Lin. An identification method for nonlinear systems with colored measurement noise. Acta Automatica Sinica, 2015, 41(11): 1877?1892
[6]	Ljung L. Perspectives on system identification. Annual Reviews in Control, 2008, 34(1): 1?12
[7]	Sch?n T B, Wills A, Ninness B. System identification of nonlinear state-space models. Automatica, 2011, 47(1): 39?49 doi: 10.1016/j.automatica.2010.10.013
[8]	Billings S A. Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains. Chichester: John Wiley & Sons, 2013.
[9]	Carini A, Orcioni S, Terenzi A, Cecchi S. Nonlinear system identification using wiener basis functions and multiple-variance perfect sequences. Signal Processing, 2019, 160: 137?149 doi: 10.1016/j.sigpro.2019.02.017
[10]	Schoukens M, Tiels K. Identification of block-oriented nonlinear systems starting from linear approximations: A survey. Automatica, 2017, 85: 272?292 doi: 10.1016/j.automatica.2017.06.044
[11]	Bershad N J, Celka P, McLaughlin S. Analysis of stochastic gradient identification of Wiener-Hammerstein systems for nonlinearities with Hermite polynomial expansions. IEEE Transactions on Signal Processing, 2001, 49(5): 1060?1072 doi: 10.1109/78.917809
[12]	Valarmathi K, Devaraj D, Radhakrishnan T K. Intelligent techniques for system identification and controller tuning in pH process. Brazilian Journal of Chemical Engineering, 2009, 26(1): 99?111 doi: 10.1590/S0104-66322009000100010
[13]	Zhu Y. Distillation column identification for control using wiener model. In: Proceedings of the American Control Conference (Cat. No. 99CH36251). San Diego, USA: IEEE, 1999. 3462-3466
[14]	Schoukens J, Suykens J, Ljung L. Wiener-Hammerstein benchmark. In: Proceedings of the 15th IFAC Symposium on System Identification (SYSID 2009). Malo, France, 2009.
[15]	Hagenblad A, Ljung L, Wills A. Maximum likelihood identification of wiener models. Automatica, 2008, 44(11): 2697?2705 doi: 10.1016/j.automatica.2008.02.016
[16]	Xu W Y, Bai E W, Cho M. System identification in the presence of outliers and random noises: A compressed sensing approach. Automatica, 2014, 50(11): 2905?2911 doi: 10.1016/j.automatica.2014.10.017
[17]	Bottegal G, Castro-Garcia R, Suykens J A K. A two-experiment approach to wiener system identification. Automatica, 2018, 93: 282?289 doi: 10.1016/j.automatica.2018.03.069
[18]	Westwick D T, Schoukens J. Initial estimates of the linear subsystems of Wiener-Hammerstein models. Automatica, 2012, 48(11): 2931?2936 doi: 10.1016/j.automatica.2012.06.091
[19]	Giordano G, Gros S, Sj?berg J. An improved method for Wiener-Hammerstein system identification based on the fractional approach. Automatica, 2018, 94: 349?360 doi: 10.1016/j.automatica.2018.04.046
[20]	Liu Q, Lin W Y, Jiang S L, Chai Y, Sun L. Robust estimation of wiener models in the presence of outliers using the VB approach. IEEE Transactions on Industrial Electronics, 2021, 68(11): 11390?11399 doi: 10.1109/TIE.2020.3028806
[21]	Xie L, Yang H Z, Huang B. FIR model identification of multirate processes with random delays using EM algorithm. AIChE Journal, 2013, 59(11): 4124?4132 doi: 10.1002/aic.14147
[22]	Agamennoni G, Nieto J I, Nebot E M. Approximate inference in state-space models with heavy-tailed noise. IEEE Transactions on Signal Processing, 2012, 60(10): 5024?5037 doi: 10.1109/TSP.2012.2208106
[23]	Bishop C M. Pattern Recognition and Machine Learning. New York: Springer, 2006.
[24]	Amari S I. Natural gradient works efficiently in learning. Neural Computation, 1998, 10(2): 251?276 doi: 10.1162/089976698300017746
[25]	Amari S I. Differential geometry of curved exponential families-curvatures and information loss. The Annals of Statistics, 1982, 10(2): 357?385
[26]	Bottou L. On-line learning and stochastic approximations. On-line Learning in Neural Networks. New York: Cambridge University Press, 1999.
[27]	Robbins H, Monro S. A stochastic approximation method. The Annals of Mathematical Statistics, 1951, 22(3): 400?407 doi: 10.1214/aoms/1177729586
[28]	Hoffman M D, Blei D M, Wang C, Paisley J. Stochastic variational inference. The Journal of Machine Learning Research, 2013, 14(1): 1303?1347