Generalization of hyperbolic smoothing approach for non-smooth and non-Lipschitz functions

Nurullah Yilmaz; Ahmet Sahiner

doi:10.3934/jimo.2021170

Article Contents

2022, Volume 18, Issue 6: 4511-4526. Doi: 10.3934/jimo.2021170

This issue Previous Article Multi-objective chance-constrained blending optimization of zinc smelter under stochastic uncertainty Next Article Penalized NCP-functions for nonlinear complementarity problems and a scaling algorithm

Generalization of hyperbolic smoothing approach for non-smooth and non-Lipschitz functions

Nurullah Yilmaz^, and
Ahmet Sahiner

Department of Mathematics, Suleyman Demirel University, Isparta, Turkey

^* Corresponding author: nurullahyilmaz@sdu.edu.tr
^* Corresponding author: nurullahyilmaz@sdu.edu.tr

Received: March 31, 2021

Revised: June 30, 2021

Early access: October 2021

Published: September 2022

Abstract Full Text(HTML) Figure(9) / Table(1) Related Papers Cited by

Abstract

In this study, we concentrate on the hyperbolic smoothing technique for some sub-classes of non-smooth functions and introduce a generalization of hyperbolic smoothing technique for non-Lipschitz functions. We present some useful properties of this generalization of hyperbolic smoothing technique. In order to illustrate the efficiency of the proposed smoothing technique, we consider the regularization problems of image restoration. The regularization problem is recast by considering the generalization of hyperbolic smoothing technique and a new algorithm is developed. Finally, the minimization algorithm is applied to image restoration problems and the numerical results are reported.

Keywords:

Mathematics Subject Classification: Primary: 65K10; Secondary: 90C30, 90C90.

Citation:

FullText(HTML)

Figure 1. The graphs of smoothing functions $ \phi_1(x, \varepsilon) $ and $ \phi_2(x, \varepsilon) $ with different $ \varepsilon $ values

Download: Full-size image PowerPoint slide

Figure 2. The graphs of smoothing functions $ \phi_1(x, \varepsilon) $ and $ \phi_2(x, \varepsilon) $ with different $ q $ values

Download: Full-size image PowerPoint slide

Figure 3. The graphs of smoothing function $ \phi_3(t, \varepsilon) $ with different $ \varepsilon $ and $ r $ values

Download: Full-size image PowerPoint slide

Figure 4. The graphs of smoothing functions $ \phi_1(x, \varepsilon) $, $ \phi_2(x, \varepsilon) $ and $ \phi_3(x, \varepsilon) $

Download: Full-size image PowerPoint slide

Figure 5. The original version of images (a) Barbara, (b) Cameraman, (c) House and (d) Peppers

Download: Full-size image PowerPoint slide

Figure 6. The noisy image and denoised images

Download: Full-size image PowerPoint slide

Figure 7. The noisy image and denoised images

Download: Full-size image PowerPoint slide

Figure 8. The noisy image and denoised images

Download: Full-size image PowerPoint slide

Figure 9. The noisy image and denoised images

Download: Full-size image PowerPoint slide

Table 1. The computational results

Noisy Image		Denoised by $ \Phi_1 $			Denoised by $ \Phi_2 $			Denoised by $ \Phi_3 $			Denoised by TV
Image Name	PSNR	Iter	PSNR	Time	Iter	PSNR	Time	Iter	PSNR	Time	Iter	PSNR	Time

Barbara	$ 20.156 $	$ 98 $	$ 23.272 $	$ 10.545 $	$ 116 $	$ 23.378 $	$ 13.194 $	$ 112 $	$ \textbf{24.684} $	$ 13.757 $	$ 124 $	$ 24.661 $	$ 14.278 $
Barbara	$ 18.161 $	$ 96 $	$ 23.155 $	$ 11.121 $	$ 120 $	$ 23.152 $	$ 13.440 $	$ 89 $	$ \textbf{23.340} $	$ 9.0128 $	$ 99 $	$ 23.147 $	$ 11.554 $
Cameraman	$ 20.217 $	$ 94 $	$ 26.597 $	$ 8.8602 $	$ 111 $	$ 26.510 $	$ 12.142 $	$ 88 $	$ \textbf{26.643} $	$ 12.9414 $	$ 129 $	$ 26.497 $	$ 15.26 $
Cameraman	$ 18.149 $	$ 96 $	$ 25.126 $	$ 12.283 $	$ 119 $	$ 24.976 $	$ 13.705 $	$ 93 $	$ \textbf{25.201} $	$ 11.0368 $	$ 102 $	$ 24.943 $	$ 10.336 $
House	$ 20.143 $	$ 80 $	$ 25.877 $	$ 1.4441 $	$ 94 $	$ 26.332 $	$ 1.8774 $	$ 45 $	$ \textbf{26.806} $	$ 0.8385 $	$ 111 $	$ 26.186 $	$ 2.2779 $
House	$ 18.299 $	$ 82 $	$ 24.775 $	$ 1.3958 $	$ 100 $	$ 24.701 $	$ 1.9154 $	$ 52 $	$ \textbf{25.590} $	$ 0.7164 $	$ 88 $	$ 25.368 $	$ 1.7774 $
Peppers	$ 20.075 $	$ 98 $	$ 26.591 $	$ 8.9505 $	$ 117 $	$ 26.430 $	$ 11.302 $	$ 86 $	$ \textbf{27.496} $	$ 10.9760 $	$ 140 $	$ 27.111 $	$ 14.11 $
Peppers	$ 18.307 $	$ 99 $	$ 25.099 $	$ 10.472 $	$ 123 $	$ 25.013 $	$ 12.721 $	$ 111 $	$ \textbf{26.980} $	$ 11.7570 $	$ 108 $	$ 25.687 $	$ 11.339 $

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	A. M. Bagirov, A. Al Nuaimat and N. Sultanova, Hyperbolic smoothing function method for minimax problems, Optimization, 62 (2013), 759-782. doi: 10.1080/02331934.2012.675335.
[2]	A. M. Bagirov, B. Ordin, G. Ozturk and A. E. Xavier, An incremental clustering algorithm based on hyperbolic smoothing, Comput. Optim. Appl., 61 (2015), 219-241. doi: 10.1007/s10589-014-9711-7.
[3]	A. M. Bagirov, N. Sultanova, A. Al Nuaimat and S. Taheri, Solving minimax problems: Local smoothing versus global smoothing, Numerical Analysis and Optimization, Springer Proceedings in Mathematics an Statistic, 235 (2018), 23-43. doi: 10.1007/978-3-319-90026-1_2.
[4]	D. P. Bertsekas, Nondifferentiable optimization via approximation, Math. Programming Stud., 3 (1975), 1-25. doi: 10.1007/BFb0120696.
[5]	W. Bian and X. Chen, Smoothing neural network for constrained non-Lipschitz optimization with applications, IEEE Trans. Neural Netw. Learn. Syst., 23 (2012), 399-411. doi: 10.1109/TNNLS.2011.2181867.
[6]	W. Bian and X. Chen, Linearly constrained non-Lipschitz optimization for image restoration, SIAM J. Imaging Sci., 8 (2015), 2294-2322. doi: 10.1137/140985639.
[7]	L. Caccetta, B. Qu and G. Zhou, A globally and quadratically convergent method for absolute value equations, Comput. Optim. Appl., 48 (2011), 45-58. doi: 10.1007/s10589-009-9242-9.
[8]	X. Chen, Smoothing methods for nonsmooth, nonconvex minimization, Math. Prog. Ser. B., 134 (2012), 71-99. doi: 10.1007/s10107-012-0569-0.
[9]	C. Chen and O. L. Mangasarian, A class of smoothing functions for nonlinear and mixed complementarity problems, Comput. Optim. Appl., 5 (1996), 97-138. doi: 10.1007/BF00249052.
[10]	X. Chen, M. K. Ng and C. Zhang, Non-Lipschitz $\ell_{p}$-regularization and box constrained model for image restoration, IEEE Trans. Image Process., 21 (2012), 4709-4721. doi: 10.1109/TIP.2012.2214051.
[11]	X. Chen and W. Zhou, Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization, SIAM J. Imaging Sciences, 3 (2010), 765-790. doi: 10.1137/080740167.
[12]	F. H. Clarke, Optimization and Nonsmooth Analysis, John Wiley & Sons, New York, 1983. doi: 10.1137/1.9781611971309.
[13]	C. Grossmann, Smoothing techniques for exact penalty methods, Contemporary Mathematics, In book Panaroma of Mathematics: Pure and Applied, 658 (2016), 249-265.
[14]	Y. Huang, H. Liu and W. Cong, A note on the smoothing quadratic regularization method for non-Lipschitz optimization, Numer. Algor., 69 (2015), 863-874. doi: 10.1007/s11075-014-9929-6.
[15]	X. Jiang and Y. Zhang, A smoothing-type algorithm for absolute value equations, J. Ind. Manag. Optim., 9 (2013), 789-798. doi: 10.3934/jimo.2013.9.789.
[16]	M. Kang and M. Jung, Simultaneous image enhancement and restoration with non-convex total variation, J. Sci. Comput., 87 (2021), Paper No. 83, 46 pp. doi: 10.1007/s10915-021-01488-x.
[17]	K. C. Kiwiel, Methods of Descent for Nondifferentiable Optimization, Lecture Notes in Mathematics, Springer-Verlag, Berlin, 1985. doi: 10.1007/BFb0074500.
[18]	G. Landi, A modified Newton projection method for $\ell_1$-regularized least squares image deblurring, J. Math. Imaging Vis., 51 (2015), 195-208. doi: 10.1007/s10851-014-0514-3.
[19]	S.-J. Lian, Smoothing approximation to $l_1$ exact penalty function for inequality constrained optimization, Appl. Math. Comput., 219 (2012), 3113-3121. doi: 10.1016/j.amc.2012.09.042.
[20]	M. M. Mäkelä and P. Neitaanmäki, Nonsmooth Optimization, World Scientific, Singapore, 1992. doi: 10.1142/1493.
[21]	N. Mau Nam, L. T. H. An, D. Giles and N. Thai An, Smoothing techniques and difference of convex functions algorithms for image reconstructions, Optimization, 69 (2020), 1601-1633. doi: 10.1080/02331934.2019.1648467.
[22]	Y. Nesterov, Smooth minimization of non-smooth functions, Math. Program., 103 (2005), 127-152. doi: 10.1007/s10107-004-0552-5.
[23]	M. Nikolova, Minimizers of cost functions involving nonsmooth data-fidelity terms. Application to the processing outliers, SIAM J. Numer. Anal., 40 (2002), 965-994. doi: 10.1137/S0036142901389165.
[24]	M. Nikolova, M. K. Ng, S. Zheng and W. K. Ching, Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization, SIAM J. Imaging Sci., 1 (2008), 2-25. doi: 10.1137/070692285.
[25]	O. N. Onak, Y. Serinagaoglu-Dogrusoz and G.-W. Weber, Effects of a priori parameter selection in minimum relative entropy method on inverse electrocardiography problem, Inverse Probl. Sci. Eng., 26 (2018), 877-897. doi: 10.1080/17415977.2017.1369979.
[26]	C. T. Pham, G. Gamard, A. Kopylov and T. T. T. Tran, An algorithm for image restoration with mixed noise using total variation regularization, Turk. J. Elec. Eng. Comp. Sci., 26 (2018), 2831-2845. doi: 10.3906/elk-1803-100.
[27]	C. T. Pham, T. T. T. Tran and G. Gamard, An efficient total variation minimization method for image restoration, Informatica, 31 (2020), 539-560. doi: 10.15388/20-INFOR407.
[28]	R. A. Polyak, Smooth optimization methods for minimax problems, SIAM J. Control Optim., 26 (1988), 1274-1286. doi: 10.1137/0326071.
[29]	L. Qi and D. Sun, Smoothing functions and smoothing Newton method for complementarity and variational inequality problems, J. Optim. Theory Appl., 113 (2002), 121-147. doi: 10.1023/A:1014861331301.
[30]	R. T. Rockefellar and R. J.-B. Wets, Variational Analysis, Springer, Berlin, 1998. doi: 10.1007/978-3-642-02431-3.
[31]	L. I. Rudin, S. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms, Physica D, 60 (1992), 259-268. doi: 10.1016/0167-2789(92)90242-F.
[32]	B. Saheya, C.-H. Yu and J.-S. Chen, Numerical comparisons based on four smoothing functions for absolute value equations, J. Appl. Math. Comput., 56 (2018), 131-149. doi: 10.1007/s12190-016-1065-0.
[33]	A. Sahiner, G. Kapusuz and N. Yilmaz, A new smoothing approach to exact penalty functions for inequality constrained optimization problems, Numer. Algebra Contol Optim., 6 (2016), 161-173. doi: 10.3934/naco.2016006.
[34]	M. Souza, A. E. Xavier, C. Lavor and N. Maculan, Hyperbolic smoothing and penalty techniques applied to molecular structure determination, Oper. Res. Lett., 39 (2011), 461-465. doi: 10.1016/j.orl.2011.07.007.
[35]	P. Taylan, G.-W. Weber and F. Yerlikaya-Ozkurt, A new approach to multivariate adaptive regression splines by using Tikhonov regularization and continuous optimization, TOP, 18 (2010), 377-395. doi: 10.1007/s11750-010-0155-7.
[36]	A. H. Tor, Hyperbolic smoothing method for sum-max problems, Neural, Parallel Sci. Comput., 24 (2016), 381-391.
[37]	S. Voronin, G. Ozkaya and D. Yoshida, Convolution based smooth approximations to the absolute value function with application to non-smooth regularization, Preprint, (2015). arXiv: 1408.6795.
[38]	C. Wu, J. Zhan, Y. Lu and J.-S. Chen, Signal reconstruction by conjugate gradient algorithm based on smoothing $l_1$-norm, Calcolo, 56 (2019), Paper No. 42, 26 pp. doi: 10.1007/s10092-019-0340-5.
[39]	A. E. Xavier, Penalizacao Hiperbolica, I Congresso Latino-Americano de Pesquisa Operacional e Engenharia de Sistemas, 8 a 11 de Novembro, Rio de Janeiro, Brasil, 1982.
[40]	A. E. Xavier, The hyperbolic smoothing clustering method, Patt. Recog., 43 (2010), 731-737. doi: 10.1016/j.patcog.2009.06.018.
[41]	A. E. Xavier and A. A. F. de Oliveira, Optimal covering of plane domains by circles via hyperbolic smoothing, J. Global Optim., 31 (2005), 493-504. doi: 10.1007/s10898-004-0737-8.
[42]	A. E. Xavier and V. L. Xavier, Solving the minimum sum-of-squares clustering problem by hyperbolic smoothing and partition into boundary and gravitational regions, Patt. Recog., 44 (2011), 70-77. doi: 10.1016/j.patcog.2010.07.004.
[43]	V. L. Xavier and A. E. Xavier, Accelerated hyperbolic smoothing method for solving the multisource Fermat-Weber and k-Median problems, Knowl. Based Syst., 191 (2020), 105226. doi: 10.1016/j.knosys.2019.105226.
[44]	N. Yilmaz and A. Sahiner, On a new smoothing technique for non-smooth, non-convex optimization, Numer. Algebra Contol Optim., 10 (2020), 317-330. doi: 10.3934/naco.2020004.
[45]	H. Yin, An adaptive smoothing method for continuous minimax problems, Asia-Pac. J. Oper. Res., 32 (2015), 1540001, 19 pp. doi: 10.1142/S0217595915400011.
[46]	L. Yuan, C. Fei, Z. Wan, W. Li and W. Wang, A nonmonotone smoothing Newton method for system of nonlinear inequalities based on a new smoothing function, Comput. Appl. Math., 38 (2019), Paper No. 91, 11 pp. doi: 10.1007/s40314-019-0856-y.
[47]	I. Zang, A smoothing out technique for min-max optimization, Math. Programm., 19 (1980), 61-77. doi: 10.1007/BF01581628.