Error analysis on regularized regression based on the Maximum correntropy criterion

Bingzheng Li; Zhengzhan Dai

doi:10.3934/mfc.2020003

Article Contents

2020, Volume 3, Issue 1: 25-40. Doi: 10.3934/mfc.2020003

This issue Previous Article Orbital stability of periodic traveling wave solutions to the coupled compound KdV and MKdV equations with two components Next Article Asymptotic expansions and Voronovskaja type theorems for the multivariate neural network operators

Error analysis on regularized regression based on the Maximum correntropy criterion

Bingzheng Li^, and
Zhengzhan Dai

School of Mathematical Science, Zhejiang University, Hangzhou, 310027, China

^* Corresponding author: Bingzheng Li
^* Corresponding author: Bingzheng Li

Received: November 30, 2019

Published: February 2020

Abstract Full Text(HTML) Related Papers Cited by

Abstract

This paper aims at the regularized learning algorithm for regression associated with the correntropy induced losses in reproducing kernel Hilbert spaces. The main target is the error analysis for the regression problem in learning theory based on the maximum correntropy. Explicit learning rates are provided. From our analysis, when choosing a suitable parameter of the loss function, we obtain satisfactory learning rates. The rates depend on the regularization error and the covering numbers of the reproducing kernel Hilbert space.

Keywords:

Mathematics Subject Classification: 41A25, 68Q32.

Citation:

Full Text(HTML)

Related Papers

Cited by

References

[1]	R. J. Bessa, V. Miranda and J. Gama, Entropy and correntropy against minimum square in offline and online three-day ahead wing power forecasting, Power Systems, IEEE Transactions on, 24 (2009), 1657-1666.
[2]	F. Cucker and D. X. Zhou, Learning Theory: An Approximation Theory Viewpoint, Cambridge University Press, 2007. doi: 10.1017/CBO9780511618796.
[3]	J. Fan, T. Hu, Q. Wu and D. X. Zhou, Consistency analysis of an empirical minimum error entropy algorithm,, Applied and Computational Harmonic Analysis, 41 (2016), 164-189. doi: 10.1016/j.acha.2014.12.005.
[4]	Y. L. Feng, J. Fang and J. A. K. Suykens, A statistical learning approach to modal regression, Journal of Machine Learning Research, 2020.
[5]	Y. L. Feng, X. L. Huang, L. Shi and J. A. K. Suykens, Learning with the maximum correntropy criterion induced losses for regression, Journal of Machine Learning Research, 16 (2015), 993-1034.
[6]	Y. L. Feng and Y. M. Ying, Learning with correntropy-induced losses for regression with mixture of symmetric stable noise, Applied and Computational Harmonic Analysis, 48 (2020), 795-810. doi: 10.1016/j.acha.2019.09.001.
[7]	R. He, W. S. Zheng and B. G. Hu, Maximum correntropy criterion for robust face recognition, IEEE Transactions on pattern Analysis and Machine Intelligence, 33 (2011), 1561-1576.
[8]	R. He, W. S. Zheng, B. G. Hu and X. W. Kong, A regularized correntropy framework for robust pattern recognition., Neural Computation, 23 (2011), 2074-2100. doi: 10.1162/NECO_a_00155.
[9]	P. J. Huber, Robust Statistics, John Wiley & Sons, 1981.
[10]	T. Hu, J. Fan, Q. Wu and D. X. Zhou, Learning theory approach to minimum error entropy criterion, Journal of Machine Learning Research, 14 (2013), 377-397.
[11]	T. Hu, J. Fan, Q. Wu and D. X. Zhou, Regularization schemes for minimum error entropy principle, Analysis and Applications, 13 (2015), 437-455. doi: 10.1142/S0219530514500110.
[12]	B. Z. Li, Approximation by multivariate Bernstein-Durrmeyer operators and learning rates of least-square regularized regression with multivariate polynomial kernels, J. Approx. Theory, 173 (2013), 33-55. doi: 10.1016/j.jat.2013.04.007.
[13]	W. Liu, P. P. Pokharel and J. C. Príncipe, Correntropy: Properties and application in non-gaussian signal processing, IEEE Transactions on Signal Processing, 55 (2007), 5286-5298. doi: 10.1109/TSP.2007.896065.
[14]	F. S. Lv and J. Fan, Optimal Learning with Gaussians and Correntropy Loss, Analysis and Applications, 2020. doi: 10.1142/S0219530519410124.
[15]	I. Santamaría, P. P. Pokharel and J. C. Príncipe, Generalized correlation function: Definition, properties, and application to blind equalization, IEEE Transactions on Signal Processing, 54 (2006), 2187-2197.
[16]	S. Smale and D. X. Zhou, Estimating the approximation error in learning theory., Analysis and Applications, 1 (2003), 17-41. doi: 10.1142/S0219530503000089.
[17]	I. Steinwart and C. Scovel, Fast rates for support vector machines, Lecture Notes in Computer Science, 3559 (2005), 279-294. doi: 10.1007/11503415_19.
[18]	V. Vapnik, Statistical Learning Theory, John Wiley & Sons, 1998.
[19]	E. D. Vito, L. Rosasco, A. Caponnetto and U. D. Giovannini, Learning from Examples as an Inverse Problem, Journal of Machine Learning Research, 6 (2005), 883-904.
[20]	C. Wang and D. X. Zhou, Optimal learning rates for least squares regularized regression with unbounded sampling, Journal of Complexity, 27 (2011), 55-67. doi: 10.1016/j.jco.2010.10.002.
[21]	Q. Wu, Y. Ying and D. X. Zhou, Learning rates of least-square regularized regression, Foundations of Computation Mathematics, 6 (2006), 171-192. doi: 10.1007/s10208-004-0155-9.
[22]	Q. Wu, Y. Ying and D. X. Zhou, Multi-kernel regularized classifiers, Journal of Complexity, 23 (2007), 108-134. doi: 10.1016/j.jco.2006.06.007.
[23]	Y. M. Ying and D. X. Zhou, Learnability of Gaussians with flexible variances, Journal of Machine Learning Research, 8 (2007), 249-276.
[24]	D. X. Zhou, The covering number in learning theory, Journal of Complexity, 18 (2002), 739-767. doi: 10.1006/jcom.2002.0635.
[25]	D. X. Zhou, Capacity of reproducing kernel spaces in learning theory, IEEE Transactions on Information Theory, 49 (2003), 1743-1752. doi: 10.1109/TIT.2003.813564.