# American Institute of Mathematical Sciences

December  2017, 11(6): 997-1025. doi: 10.3934/ipi.2017046

## Analysis of a variational model for motion compensated inpainting

 1 Istituto per le Applicazioni del Calcolo, CNR, Via dei Taurini 19,00185 Roma, Italy 2 Dipartimento di Matematica e Informatica, Universitá della Calabria, Via Pietro Bucci, Arcavacata di Rende, 87036 Cosenza, Italy

Received  October 2016 Revised  June 2017 Published  September 2017

We study a variational problem for simultaneous video inpainting and motion estimation. We consider a functional proposed by Lauze and Nielsen [25] and we study, by means of the relaxation method of the Calculus of Variations, a slightly modified version of this functional. The domain of the relaxed functional is constituted of functions of bounded variation and we compute a representation formula of the relaxed functional. The representation formula shows the role of discontinuities of the various functions involved in the variational model. The present study clarifies the variational properties of the functional proposed in [25] for motion compensated video inpainting.

Citation: Riccardo March, Giuseppe Riey. Analysis of a variational model for motion compensated inpainting. Inverse Problems and Imaging, 2017, 11 (6) : 997-1025. doi: 10.3934/ipi.2017046
##### References:
 [1] L. Ambrosio, Variational problems in SBV and image segmentation, Acta Appl. Math., 17 (1989), 1-40.  doi: 10.1007/BF00052492. [2] L. Ambrosio, N. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, Oxford University Press, New York, 2000. [3] L. Ambrosio and D. Pallara, Integral representation of relaxed functionals on $BV(\mathbb{R}^n, \mathbb{R}^k)$ and polyhedral approximation, Indiana Univ. Math. J., 42 (1993), 295-321.  doi: 10.1512/iumj.1993.42.42015. [4] G. Aubert, R. Deriche and P. Kornprobst, Computing optimal flow via variational techniques, SIAM J. App. Math., 60 (2000), 156-182.  doi: 10.1137/S0036139998340170. [5] G. Aubert and P. Kornprobst, A mathematical study of the relaxed optical flow problem in the space $BV(Ω)$, SIAM J. Math. Anal., 30 (1999), 1282-1308.  doi: 10.1137/S003614109834123X. [6] G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, 2nd Edition, Springer, New York, 2006. [7] M. Bertalmio, A. Bertozzi and G. Sapiro, Navier-Stokes, fluid-dynamics and image and video inpainting, in Proceedings of Computer Vision and Pattern Recognition, 2001,355-362. doi: 10.1109/CVPR.2001.990497. [8] K. Bredies, K. Kunish and T. Pock, Total generalized variation, SIAM J. Imaging Sciences, 3 (2010), 492-526.  doi: 10.1137/090769521. [9] K. Bredies, K. Kunish and T. Valkonen, Properties of $L^1-TGV^2$: The one-dimensional case, J. Math. Anal. Appl., 398 (2013), 438-454.  doi: 10.1016/j.jmaa.2012.08.053. [10] T. Brox, A. Bruhn, N. Papenberg and J. Weickert, High accuracy optical flow estimation based on a theory for warping, Proceedings of the 8th European Conference on Computer Vision (eds. T. Pajdla and J. Matas), Springer, 3024 (2004), 25-36. doi: 10.1007/978-3-540-24673-2_3. [11] A. Buffa and C. Ortner, Compact embeddings of broken Sobolev spaces and applications, IMA J. Numer. Anal., 29 (2009), 827-855.  doi: 10.1093/imanum/drn038. [12] G. Buttazzo, Semicontinuity, Relaxation and Integral Representation in the Calculus of Variations, Pitman Research Notes in Mathematics Series, Vol. 207, Longman Scientific & Technical, UK, 1989. [13] J. P. Cocquerez, L. Chanas and J. Blanc-Talon, Simultaneous inpainting and motion estimation of highly degraded video-sequences, in Scandinavian Conference on Image Analysis, LNCS, Springer-Verlag, 2749 (2003), 523-530. [14] S. Conti, J. Ginster and M. Rumpf, A BV functional and its relaxation for joint motion estimation and image sequence recovery, ESAIM: Mathematical Modelling and Numerical Analysis, 49 (2015), 1463-1487.  doi: 10.1051/m2an/2015036. [15] A. Corbo Esposito and R. De Arcangelis, Comparison results for some types of relaxation of variational integral functionals, Ann. Mat. Pura Appl., 164 (1993), 155-193.  doi: 10.1007/BF01759320. [16] F. Demengel, Fonctions á hessien borné, Annales de l'Institut Fourier, 34 (1984), 155-190. [17] F. Demengel and R. Temam, Convex functions of a measure and applications, Indiana University Mathematics Journal, 33 (1984), 673-709.  doi: 10.1512/iumj.1984.33.33036. [18] M. Giaquinta, G. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅰ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 143-156. [19] M. Giaquinta, G. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅱ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 157-172. [20] E. Giusti, Minimal Surfaces and Functions of Bounded Variation, Birkhäuser, Boston, 1984.  doi: 10.1007/978-1-4684-9486-0. [21] C. Goffman and J. Serrin, Sublinear functions of measures and variationals integrals, Duke Math. J., 31 (1964), 159-178. [22] B. K. P. Horn and B. G. Schunck, Determining optical flow, Artificial Intelligence, 17 (1981), 185-203. [23] S. H. Keller, F. Lauze and M. Nielsen, Deintarlacing using variational methods, IEEE Trans. Image Proc., 17 (2008), 2015-2028.  doi: 10.1109/TIP.2008.2003394. [24] S. H. Keller, F. Lauze and M. Nielsen, Video super-resolution using simultaneous motion and intensity calculations, IEEE Trans. Image Proc., 20 (2011), 1870-1884.  doi: 10.1109/TIP.2011.2106793. [25] F. Lauze and M. Nielsen, A Variational algorithm for motion compensated inpainting, in British Machine Vision Conference (eds. S. Barman, A. Hoppe and T. Ellis editors), BMVA, 2 (2004), 777-787. [26] Y. G. Reshetnyak, Weak convergence of completely additive vector functions on a set, (Russian) Sibirski Mat. Zh., 9 (1968), 1386-1394; translation in Siberian Math. J., 9 (1968), 1039-1045. [27] S. Uras, F. Girosi, A. Verri and V. Torre, A computational approach to motion perception, Biol. Cybern., 60 (1988), 79-87. [28] W. P. Ziemer, Weakly differentiable functions, Springer-Verlag, New York, 1989.  doi: 10.1007/978-1-4612-1015-3.

show all references

##### References:
 [1] L. Ambrosio, Variational problems in SBV and image segmentation, Acta Appl. Math., 17 (1989), 1-40.  doi: 10.1007/BF00052492. [2] L. Ambrosio, N. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, Oxford University Press, New York, 2000. [3] L. Ambrosio and D. Pallara, Integral representation of relaxed functionals on $BV(\mathbb{R}^n, \mathbb{R}^k)$ and polyhedral approximation, Indiana Univ. Math. J., 42 (1993), 295-321.  doi: 10.1512/iumj.1993.42.42015. [4] G. Aubert, R. Deriche and P. Kornprobst, Computing optimal flow via variational techniques, SIAM J. App. Math., 60 (2000), 156-182.  doi: 10.1137/S0036139998340170. [5] G. Aubert and P. Kornprobst, A mathematical study of the relaxed optical flow problem in the space $BV(Ω)$, SIAM J. Math. Anal., 30 (1999), 1282-1308.  doi: 10.1137/S003614109834123X. [6] G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, 2nd Edition, Springer, New York, 2006. [7] M. Bertalmio, A. Bertozzi and G. Sapiro, Navier-Stokes, fluid-dynamics and image and video inpainting, in Proceedings of Computer Vision and Pattern Recognition, 2001,355-362. doi: 10.1109/CVPR.2001.990497. [8] K. Bredies, K. Kunish and T. Pock, Total generalized variation, SIAM J. Imaging Sciences, 3 (2010), 492-526.  doi: 10.1137/090769521. [9] K. Bredies, K. Kunish and T. Valkonen, Properties of $L^1-TGV^2$: The one-dimensional case, J. Math. Anal. Appl., 398 (2013), 438-454.  doi: 10.1016/j.jmaa.2012.08.053. [10] T. Brox, A. Bruhn, N. Papenberg and J. Weickert, High accuracy optical flow estimation based on a theory for warping, Proceedings of the 8th European Conference on Computer Vision (eds. T. Pajdla and J. Matas), Springer, 3024 (2004), 25-36. doi: 10.1007/978-3-540-24673-2_3. [11] A. Buffa and C. Ortner, Compact embeddings of broken Sobolev spaces and applications, IMA J. Numer. Anal., 29 (2009), 827-855.  doi: 10.1093/imanum/drn038. [12] G. Buttazzo, Semicontinuity, Relaxation and Integral Representation in the Calculus of Variations, Pitman Research Notes in Mathematics Series, Vol. 207, Longman Scientific & Technical, UK, 1989. [13] J. P. Cocquerez, L. Chanas and J. Blanc-Talon, Simultaneous inpainting and motion estimation of highly degraded video-sequences, in Scandinavian Conference on Image Analysis, LNCS, Springer-Verlag, 2749 (2003), 523-530. [14] S. Conti, J. Ginster and M. Rumpf, A BV functional and its relaxation for joint motion estimation and image sequence recovery, ESAIM: Mathematical Modelling and Numerical Analysis, 49 (2015), 1463-1487.  doi: 10.1051/m2an/2015036. [15] A. Corbo Esposito and R. De Arcangelis, Comparison results for some types of relaxation of variational integral functionals, Ann. Mat. Pura Appl., 164 (1993), 155-193.  doi: 10.1007/BF01759320. [16] F. Demengel, Fonctions á hessien borné, Annales de l'Institut Fourier, 34 (1984), 155-190. [17] F. Demengel and R. Temam, Convex functions of a measure and applications, Indiana University Mathematics Journal, 33 (1984), 673-709.  doi: 10.1512/iumj.1984.33.33036. [18] M. Giaquinta, G. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅰ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 143-156. [19] M. Giaquinta, G. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅱ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 157-172. [20] E. Giusti, Minimal Surfaces and Functions of Bounded Variation, Birkhäuser, Boston, 1984.  doi: 10.1007/978-1-4684-9486-0. [21] C. Goffman and J. Serrin, Sublinear functions of measures and variationals integrals, Duke Math. J., 31 (1964), 159-178. [22] B. K. P. Horn and B. G. Schunck, Determining optical flow, Artificial Intelligence, 17 (1981), 185-203. [23] S. H. Keller, F. Lauze and M. Nielsen, Deintarlacing using variational methods, IEEE Trans. Image Proc., 17 (2008), 2015-2028.  doi: 10.1109/TIP.2008.2003394. [24] S. H. Keller, F. Lauze and M. Nielsen, Video super-resolution using simultaneous motion and intensity calculations, IEEE Trans. Image Proc., 20 (2011), 1870-1884.  doi: 10.1109/TIP.2011.2106793. [25] F. Lauze and M. Nielsen, A Variational algorithm for motion compensated inpainting, in British Machine Vision Conference (eds. S. Barman, A. Hoppe and T. Ellis editors), BMVA, 2 (2004), 777-787. [26] Y. G. Reshetnyak, Weak convergence of completely additive vector functions on a set, (Russian) Sibirski Mat. Zh., 9 (1968), 1386-1394; translation in Siberian Math. J., 9 (1968), 1039-1045. [27] S. Uras, F. Girosi, A. Verri and V. Torre, A computational approach to motion perception, Biol. Cybern., 60 (1988), 79-87. [28] W. P. Ziemer, Weakly differentiable functions, Springer-Verlag, New York, 1989.  doi: 10.1007/978-1-4612-1015-3.
 [1] Bernard Dacorogna, Giovanni Pisante, Ana Margarida Ribeiro. On non quasiconvex problems of the calculus of variations. Discrete and Continuous Dynamical Systems, 2005, 13 (4) : 961-983. doi: 10.3934/dcds.2005.13.961 [2] Daniel Faraco, Jan Kristensen. Compactness versus regularity in the calculus of variations. Discrete and Continuous Dynamical Systems - B, 2012, 17 (2) : 473-485. doi: 10.3934/dcdsb.2012.17.473 [3] Felix Sadyrbaev. Nonlinear boundary value problems of the calculus of variations. Conference Publications, 2003, 2003 (Special) : 760-770. doi: 10.3934/proc.2003.2003.760 [4] Agnieszka B. Malinowska, Delfim F. M. Torres. Euler-Lagrange equations for composition functionals in calculus of variations on time scales. Discrete and Continuous Dynamical Systems, 2011, 29 (2) : 577-593. doi: 10.3934/dcds.2011.29.577 [5] Delfim F. M. Torres. Proper extensions of Noether's symmetry theorem for nonsmooth extremals of the calculus of variations. Communications on Pure and Applied Analysis, 2004, 3 (3) : 491-500. doi: 10.3934/cpaa.2004.3.491 [6] Nuno R. O. Bastos, Rui A. C. Ferreira, Delfim F. M. Torres. Necessary optimality conditions for fractional difference problems of the calculus of variations. Discrete and Continuous Dynamical Systems, 2011, 29 (2) : 417-437. doi: 10.3934/dcds.2011.29.417 [7] Jacky Cresson, Fernando Jiménez, Sina Ober-Blöbaum. Continuous and discrete Noether's fractional conserved quantities for restricted calculus of variations. Journal of Geometric Mechanics, 2022, 14 (1) : 57-89. doi: 10.3934/jgm.2021012 [8] Kazuo Aoki, Ansgar Jüngel, Peter A. Markowich. Small velocity and finite temperature variations in kinetic relaxation models. Kinetic and Related Models, 2010, 3 (1) : 1-15. doi: 10.3934/krm.2010.3.1 [9] Nikos Katzourakis. Nonuniqueness in vector-valued calculus of variations in $L^\infty$ and some Linear elliptic systems. Communications on Pure and Applied Analysis, 2015, 14 (1) : 313-327. doi: 10.3934/cpaa.2015.14.313 [10] Nikos Katzourakis. Corrigendum to the paper: Nonuniqueness in Vector-Valued Calculus of Variations in $L^\infty$ and some Linear Elliptic Systems. Communications on Pure and Applied Analysis, 2019, 18 (4) : 2197-2198. doi: 10.3934/cpaa.2019098 [11] Ioan Bucataru, Matias F. Dahl. Semi-basic 1-forms and Helmholtz conditions for the inverse problem of the calculus of variations. Journal of Geometric Mechanics, 2009, 1 (2) : 159-180. doi: 10.3934/jgm.2009.1.159 [12] Gisella Croce, Nikos Katzourakis, Giovanni Pisante. $\mathcal{D}$-solutions to the system of vectorial Calculus of Variations in $L^∞$ via the singular value problem. Discrete and Continuous Dynamical Systems, 2017, 37 (12) : 6165-6181. doi: 10.3934/dcds.2017266 [13] Hans Josef Pesch. Carathéodory's royal road of the calculus of variations: Missed exits to the maximum principle of optimal control theory. Numerical Algebra, Control and Optimization, 2013, 3 (1) : 161-173. doi: 10.3934/naco.2013.3.161 [14] Ivar Ekeland. From Frank Ramsey to René Thom: A classical problem in the calculus of variations leading to an implicit differential equation. Discrete and Continuous Dynamical Systems, 2010, 28 (3) : 1101-1119. doi: 10.3934/dcds.2010.28.1101 [15] Peter Frolkovič, Viera Kleinová. A new numerical method for level set motion in normal direction used in optical flow estimation. Discrete and Continuous Dynamical Systems - S, 2021, 14 (3) : 851-863. doi: 10.3934/dcdss.2020347 [16] Pamela A. Marshall, Eden E. Tanzosh, Francisco J. Solis, Haiyan Wang. Response of yeast mutants to extracellular calcium variations. Discrete and Continuous Dynamical Systems - B, 2009, 12 (2) : 439-453. doi: 10.3934/dcdsb.2009.12.439 [17] Fabrizio Colombo, Graziano Gentili, Irene Sabadini and Daniele C. Struppa. A functional calculus in a noncommutative setting. Electronic Research Announcements, 2007, 14: 60-68. doi: 10.3934/era.2007.14.60 [18] Ken Abe. Some uniqueness result of the Stokes flow in a half space in a space of bounded functions. Discrete and Continuous Dynamical Systems - S, 2014, 7 (5) : 887-900. doi: 10.3934/dcdss.2014.7.887 [19] Vladimir V. Kisil. Mobius transformations and monogenic functional calculus. Electronic Research Announcements, 1996, 2: 26-33. [20] Hamed Azizollahi, Marion Darbas, Mohamadou M. Diallo, Abdellatif El Badia, Stephanie Lohrengel. EEG in neonates: Forward modeling and sensitivity analysis with respect to variations of the conductivity. Mathematical Biosciences & Engineering, 2018, 15 (4) : 905-932. doi: 10.3934/mbe.2018041

2020 Impact Factor: 1.639