April  2022, 9(2): 123-150. doi: 10.3934/jdg.2021031

Decision Theory and large deviations for dynamical hypotheses tests: The Neyman-Pearson Lemma, Min-Max and Bayesian tests

IME - UFRGS, Av. Bento Goncalves 9500 - 91509-900 - Porto Alegre, Brazil

*Corresponding author: Artur O. Lopes

Received  January 2021 Revised  August 2021 Published  April 2022 Early access  January 2022

Fund Project: H.H. Ferreira was supported by CAPES-Brazil scholarship. A.O. Lopes and S.R.C. Lopes were partially supported by grant of CNPq-Brazil

We analyze hypotheses tests using classical results on large deviations to compare two models, each one described by a different Hölder Gibbs probability measure. One main difference to the classical hypothesis tests in Decision Theory is that here the two measures are singular with respect to each other. Among other objectives, we are interested in the decay rate of the wrong decisions probability, when the sample size $ n $ goes to infinity. We show a dynamical version of the Neyman-Pearson Lemma displaying the ideal test within a certain class of similar tests. This test becomes exponentially better, compared to other alternative tests, when the sample size goes to infinity. We are able to present the explicit exponential decay rate. We also consider both, the Min-Max and a certain type of Bayesian hypotheses tests. We shall consider these tests in the log likelihood framework by using several tools of Thermodynamic Formalism. Versions of the Stein's Lemma and Chernoff's information are also presented.

Citation: Hermes H. Ferreira, Artur O. Lopes, Silvia R. C. Lopes. Decision Theory and large deviations for dynamical hypotheses tests: The Neyman-Pearson Lemma, Min-Max and Bayesian tests. Journal of Dynamics and Games, 2022, 9 (2) : 123-150. doi: 10.3934/jdg.2021031
References:
[1] F. Abramovich and Y. Ritov, Statistical Theory: A Concise Introduction, Boca Raton, CRC Press, 2013.  doi: 10.1201/b14755.
[2]

R. R. Bahadur, Large deviations of the maximum likelihood estimate in the Markov chain case, In J. S. Rostag, M. H. Rizvi and D. Siegmund, editors, Recent Advances in Statistics, 273-283. Boston, Academic Press, 1983. doi: 10.1016/B978-0-12-589320-6.50017-4.

[3]

A. Baraviera, R. Leplaideur and A. O. Lopes, Ergodic Optimization, Zero Temperature and the Max-Plus Algebra, $23^{\text{o}}$ Colóquio Brasileiro de Matemática, IMPA, Rio de Janeiro, 2013.

[4]

M. Barni and B. Tondi, Binary hypothesis testing game with training data, IEEE Transactions on Information Theory, 60 (2014), 4848-4866.  doi: 10.1109/TIT.2014.2325571.

[5]

T. BenoistV. JakšićY. Pautrat and C.-A. Pillet, On entropy production of repeated quantum measurements I. General theory, Commun. Math. Phys., 357 (2018), 77-123.  doi: 10.1007/s00220-017-2947-1.

[6]

D. Blackwell and M. A. Girshick, Theory of Games and Statistical Decisions, Dover publications, 1979.

[7]

D. Bohle, A. Marynych and M. Meiners, A fundamental problem of hypotesis testing with finite e-commerce, Appl. Stoch. Models Bus. Ind., 37 (2021), 454-474. preprint, arXiv: 2006.05786. doi: 10.1002/asmb.2574.

[8] L. D. Broemeling, Bayesian Inference for Stochastic Processes, Boca Raton, CRC Press, 2018. 
[9]

J. A. Bucklew, Large Deviation Techniques in Decision, Simulation and Estimation, New York, Wiley, 1990.

[10]

A. Caticha, Lectures on probability, entropy and statistical physics, Entropic Physics, preprint, arXiv: 0808.0012.

[11]

J.-R. Chazottes and D. Gabrielli, Large deviations for empirical entropies of g-measures, Nonlinearity, 18 (2005), 2545-2563.  doi: 10.1088/0951-7715/18/6/007.

[12] T. M. Cover and J. A. Thomas, Elements of Information Theory, second edition, New York, Wiley Press, 2006. 
[13]

G. B. CybisS. R. C. Lopes and H. P. Pinheiro, Power of the likelihood ratio test for models of DNA base substitution, Journal of Applied Statistics, 38 (2011), 2723-2737.  doi: 10.1080/02664763.2011.567253.

[14]

R. DakovicM. Denker and M. Gordin, Circular unitary ensembles: Parametric models and their asymptotic maximum likelihood estimates, Journal of Mathematical Sciences, 219 (2016), 714-730.  doi: 10.1007/s10958-016-3141-2.

[15]

A. Dembo and O. Zeitouni, Large Deviation Techniques and Applications, New York, Springer Verlag, 2010. doi: 10.1007/978-3-642-03311-7.

[16]

M. Denker, Basics of Thermodynamics, Lecture Notes - Penn State Univ., 2011.

[17]

M. Denker and W. Woyczynski, Introductory Statistics and Random Phenomena: Uncertainty, Complexity and Chaotic Behavior in Engineering and Science, New York, Birkhäuser, 2012. doi: 10.1007/978-3-319-66152-0.

[18]

R. S. Ellis, Entropy, Large Deviations, and Statistical Mechanics, New York, Springer Verlag, 2006. doi: 10.1007/3-540-29060-5.

[19]

H. H. Ferreira, A. O. Lopes and E. R. Oliveira, An iteration process for approximating subactions, Modeling, Dynamics, Optimization and Bioeconomics IV, Editors: Alberto Pinto and David Zilberman, Springer Proceedings in Mathematics and Statistics, New York, Springer Verlag (2021), 187-212.

[20]

V. GirardinL. Lhote and P. Regnault, Different closed-form expressions for generalized entropy rates of Markov chains, Methodology and Computing in Applied Probability, 21 (2019), 1431-1452.  doi: 10.1007/s11009-018-9679-3.

[21]

V. Girardin and P. Regnault, Escort distributions minimizing the Kullback-Leibler divergence for a large deviations principle and tests of entropy level, Ann Inst Stat Math., 68 (2016), 439-468.  doi: 10.1007/s10463-014-0501-x.

[22]

M. J. KarlingS. R. C. Lopes and R. M. de Souza, A Bayesian approach for estimating the parameters of an $\alpha$-stable distribution, Journal of Statistical Computation and Simulation, 91 (2021), 1713-1748.  doi: 10.1080/00949655.2020.1865958.

[23]

Y. Kifer, Large deviations in dynamical systems and stochastic processes, Trans. Amer. Math. Soc., 321 (1990), 505-524.  doi: 10.1090/S0002-9947-1990-1025756-7.

[24]

A. Lopes, Entropy, pressure and large deviation, Cellular Automata, Dynamical Systems and Neural Networks, E. Goles e S. Martinez (eds.), Kluwer, Massachusets, (1994), 79-146.

[25]

A. O. Lopes, Thermodynamic formalism, maximizing probabilities and large deviations, Preprint - UFRGS.

[26]

A. O. Lopes, Entropy and large deviation, NonLinearity, 3 (1990), 527-546.  doi: 10.1088/0951-7715/3/2/013.

[27]

A. O. Lopes, S. R. C. Lopes and P. Varandas, Bayes posterior convergence for loss functions via almost additive thermodynamic formalism, to appear in Journ. of Statis. Physics.

[28]

A. O. Lopes and J. K. Mengue, On information gain, Kullback-Leibler divergence, entropy production and the involution kernel, to appear in Disc. and Cont. Dyn. Syst. Series A.

[29]

A. O. Lopes and R. Ruggiero, Nonequilibrium in thermodynamic formalism: The second law, gases and information geometry, Qualitative Theory of Dynamical Systems, 21 (2022). doi: 10.1007/s12346-021-00551-0.

[30]

K. McGoff, S. Mukherjee and A. Nobel, Gibbs posterior convergence and thermodynamic formalism, to appear in Adv. in Appl. Prob.

[31]

W. Parry and M. Pollicott, Zeta functions and the periodic orbit structure of hyperbolic dynamics, Astérisque, (1990), 187-188.

[32]

V. K. Rohatgi, An Introduction to Probability Theory and Mathematical Statistics, New York: Wiley, 1976.

[33]

T. Sagawa, Entropy, divergence and majorization in classical and quantum theory, arXiv: 2007.09974.

[34] Y. Suhov and M. Kelbert, Probability and Statistics by Example. I, Cambridge, Cambridge University Press, 2014.  doi: 10.1017/CBO9781139087773.
[35]

D. A. van Dyk, The Role of statistics in the discovery of a higgs boson, Annual Review of Statistics and Its Application, 1 (2014), 41-59.  doi: 10.1146/annurev-statistics-062713-085841.

[36]

A. C. D. van Enter, A. O. Lopes, S. R. C Lopes and J. K. Mengue, How to get the Bayesian a posteriori probability from an a priori probability via thermodynamic formalism for plans; the connection to disordered systems, work in progress.

[37] W. von der LindenV. Dose and U. von Toussaint, Bayesian Probability Theory Applications in the Physical Sciences, Cambridge, Cambridge University Press, 2014.  doi: 10.1017/CBO9781139565608.

show all references

References:
[1] F. Abramovich and Y. Ritov, Statistical Theory: A Concise Introduction, Boca Raton, CRC Press, 2013.  doi: 10.1201/b14755.
[2]

R. R. Bahadur, Large deviations of the maximum likelihood estimate in the Markov chain case, In J. S. Rostag, M. H. Rizvi and D. Siegmund, editors, Recent Advances in Statistics, 273-283. Boston, Academic Press, 1983. doi: 10.1016/B978-0-12-589320-6.50017-4.

[3]

A. Baraviera, R. Leplaideur and A. O. Lopes, Ergodic Optimization, Zero Temperature and the Max-Plus Algebra, $23^{\text{o}}$ Colóquio Brasileiro de Matemática, IMPA, Rio de Janeiro, 2013.

[4]

M. Barni and B. Tondi, Binary hypothesis testing game with training data, IEEE Transactions on Information Theory, 60 (2014), 4848-4866.  doi: 10.1109/TIT.2014.2325571.

[5]

T. BenoistV. JakšićY. Pautrat and C.-A. Pillet, On entropy production of repeated quantum measurements I. General theory, Commun. Math. Phys., 357 (2018), 77-123.  doi: 10.1007/s00220-017-2947-1.

[6]

D. Blackwell and M. A. Girshick, Theory of Games and Statistical Decisions, Dover publications, 1979.

[7]

D. Bohle, A. Marynych and M. Meiners, A fundamental problem of hypotesis testing with finite e-commerce, Appl. Stoch. Models Bus. Ind., 37 (2021), 454-474. preprint, arXiv: 2006.05786. doi: 10.1002/asmb.2574.

[8] L. D. Broemeling, Bayesian Inference for Stochastic Processes, Boca Raton, CRC Press, 2018. 
[9]

J. A. Bucklew, Large Deviation Techniques in Decision, Simulation and Estimation, New York, Wiley, 1990.

[10]

A. Caticha, Lectures on probability, entropy and statistical physics, Entropic Physics, preprint, arXiv: 0808.0012.

[11]

J.-R. Chazottes and D. Gabrielli, Large deviations for empirical entropies of g-measures, Nonlinearity, 18 (2005), 2545-2563.  doi: 10.1088/0951-7715/18/6/007.

[12] T. M. Cover and J. A. Thomas, Elements of Information Theory, second edition, New York, Wiley Press, 2006. 
[13]

G. B. CybisS. R. C. Lopes and H. P. Pinheiro, Power of the likelihood ratio test for models of DNA base substitution, Journal of Applied Statistics, 38 (2011), 2723-2737.  doi: 10.1080/02664763.2011.567253.

[14]

R. DakovicM. Denker and M. Gordin, Circular unitary ensembles: Parametric models and their asymptotic maximum likelihood estimates, Journal of Mathematical Sciences, 219 (2016), 714-730.  doi: 10.1007/s10958-016-3141-2.

[15]

A. Dembo and O. Zeitouni, Large Deviation Techniques and Applications, New York, Springer Verlag, 2010. doi: 10.1007/978-3-642-03311-7.

[16]

M. Denker, Basics of Thermodynamics, Lecture Notes - Penn State Univ., 2011.

[17]

M. Denker and W. Woyczynski, Introductory Statistics and Random Phenomena: Uncertainty, Complexity and Chaotic Behavior in Engineering and Science, New York, Birkhäuser, 2012. doi: 10.1007/978-3-319-66152-0.

[18]

R. S. Ellis, Entropy, Large Deviations, and Statistical Mechanics, New York, Springer Verlag, 2006. doi: 10.1007/3-540-29060-5.

[19]

H. H. Ferreira, A. O. Lopes and E. R. Oliveira, An iteration process for approximating subactions, Modeling, Dynamics, Optimization and Bioeconomics IV, Editors: Alberto Pinto and David Zilberman, Springer Proceedings in Mathematics and Statistics, New York, Springer Verlag (2021), 187-212.

[20]

V. GirardinL. Lhote and P. Regnault, Different closed-form expressions for generalized entropy rates of Markov chains, Methodology and Computing in Applied Probability, 21 (2019), 1431-1452.  doi: 10.1007/s11009-018-9679-3.

[21]

V. Girardin and P. Regnault, Escort distributions minimizing the Kullback-Leibler divergence for a large deviations principle and tests of entropy level, Ann Inst Stat Math., 68 (2016), 439-468.  doi: 10.1007/s10463-014-0501-x.

[22]

M. J. KarlingS. R. C. Lopes and R. M. de Souza, A Bayesian approach for estimating the parameters of an $\alpha$-stable distribution, Journal of Statistical Computation and Simulation, 91 (2021), 1713-1748.  doi: 10.1080/00949655.2020.1865958.

[23]

Y. Kifer, Large deviations in dynamical systems and stochastic processes, Trans. Amer. Math. Soc., 321 (1990), 505-524.  doi: 10.1090/S0002-9947-1990-1025756-7.

[24]

A. Lopes, Entropy, pressure and large deviation, Cellular Automata, Dynamical Systems and Neural Networks, E. Goles e S. Martinez (eds.), Kluwer, Massachusets, (1994), 79-146.

[25]

A. O. Lopes, Thermodynamic formalism, maximizing probabilities and large deviations, Preprint - UFRGS.

[26]

A. O. Lopes, Entropy and large deviation, NonLinearity, 3 (1990), 527-546.  doi: 10.1088/0951-7715/3/2/013.

[27]

A. O. Lopes, S. R. C. Lopes and P. Varandas, Bayes posterior convergence for loss functions via almost additive thermodynamic formalism, to appear in Journ. of Statis. Physics.

[28]

A. O. Lopes and J. K. Mengue, On information gain, Kullback-Leibler divergence, entropy production and the involution kernel, to appear in Disc. and Cont. Dyn. Syst. Series A.

[29]

A. O. Lopes and R. Ruggiero, Nonequilibrium in thermodynamic formalism: The second law, gases and information geometry, Qualitative Theory of Dynamical Systems, 21 (2022). doi: 10.1007/s12346-021-00551-0.

[30]

K. McGoff, S. Mukherjee and A. Nobel, Gibbs posterior convergence and thermodynamic formalism, to appear in Adv. in Appl. Prob.

[31]

W. Parry and M. Pollicott, Zeta functions and the periodic orbit structure of hyperbolic dynamics, Astérisque, (1990), 187-188.

[32]

V. K. Rohatgi, An Introduction to Probability Theory and Mathematical Statistics, New York: Wiley, 1976.

[33]

T. Sagawa, Entropy, divergence and majorization in classical and quantum theory, arXiv: 2007.09974.

[34] Y. Suhov and M. Kelbert, Probability and Statistics by Example. I, Cambridge, Cambridge University Press, 2014.  doi: 10.1017/CBO9781139087773.
[35]

D. A. van Dyk, The Role of statistics in the discovery of a higgs boson, Annual Review of Statistics and Its Application, 1 (2014), 41-59.  doi: 10.1146/annurev-statistics-062713-085841.

[36]

A. C. D. van Enter, A. O. Lopes, S. R. C Lopes and J. K. Mengue, How to get the Bayesian a posteriori probability from an a priori probability via thermodynamic formalism for plans; the connection to disordered systems, work in progress.

[37] W. von der LindenV. Dose and U. von Toussaint, Bayesian Probability Theory Applications in the Physical Sciences, Cambridge, Cambridge University Press, 2014.  doi: 10.1017/CBO9781139565608.
Figure 1.  Graphs of $ P_0 $ (in solid line) and $ P_1 $ (in dashed line) for the functions defined in (3.27). For these plots we consider the data from the example in Section 7
Figure 2.  The large deviation rate function $ I_1(\cdot) $ at points $ v_1 $, $ G_1 $ and zero, where $ G_1 = E - \left(\int \log J_0 \, d \mu_{0} \, - \int \log \, J_1\, \, d\mu_0\, \right) $
Figure 3.  Graph of the function $ R(\lambda) = I_0^{E_{\lambda, \lambda}}(0) $, when $ 0\leq \lambda\leq \lambda_s $, using the stochastic matrix $ \mathcal{P}_{j} $, for $ j = 0, 1 $, from the example in Section 7
Figure 4.  Graphs of the functions $ \lambda \to \int \log J_\lambda d \mu_0 $ (in dotted line) and $ \lambda \to \int \log J_\lambda d \mu_1 $ (in dashed and dotted line) together with the graph of the values $ E_\lambda $ (in solid line), as a function of $ \lambda $, when $ 0\leq \lambda\leq \lambda_s $. The stochastic matrix $ \mathcal{P}_{j} $, for $ j = 0, 1 $, is from the example in Section 7
[1]

Onur Teymur, Sarah Filippi. A Bayesian nonparametric test for conditional independence. Foundations of Data Science, 2020, 2 (2) : 155-172. doi: 10.3934/fods.2020009

[2]

Luis Barreira. Nonadditive thermodynamic formalism: Equilibrium and Gibbs measures. Discrete and Continuous Dynamical Systems, 2006, 16 (2) : 279-305. doi: 10.3934/dcds.2006.16.279

[3]

José M. Amigó, Ángel Giménez. Formulas for the topological entropy of multimodal maps based on min-max symbols. Discrete and Continuous Dynamical Systems - B, 2015, 20 (10) : 3415-3434. doi: 10.3934/dcdsb.2015.20.3415

[4]

Jinchuan Zhou, Changyu Wang, Naihua Xiu, Soonyi Wu. First-order optimality conditions for convex semi-infinite min-max programming with noncompact sets. Journal of Industrial and Management Optimization, 2009, 5 (4) : 851-866. doi: 10.3934/jimo.2009.5.851

[5]

Roberto Livrea, Salvatore A. Marano. A min-max principle for non-differentiable functions with a weak compactness condition. Communications on Pure and Applied Analysis, 2009, 8 (3) : 1019-1029. doi: 10.3934/cpaa.2009.8.1019

[6]

Meixia Li, Changyu Wang, Biao Qu. Non-convex semi-infinite min-max optimization with noncompact sets. Journal of Industrial and Management Optimization, 2017, 13 (4) : 1859-1881. doi: 10.3934/jimo.2017022

[7]

Alvaro Sandroni, Eran Shmaya. A prequential test for exchangeable theories. Journal of Dynamics and Games, 2014, 1 (3) : 497-505. doi: 10.3934/jdg.2014.1.497

[8]

Li Li, Xinzhen Zhang, Zheng-Hai Huang, Liqun Qi. Test of copositive tensors. Journal of Industrial and Management Optimization, 2019, 15 (2) : 881-891. doi: 10.3934/jimo.2018075

[9]

Lucian Coroianu, Sorin G. Gal. New approximation properties of the Bernstein max-min operators and Bernstein max-product operators. Mathematical Foundations of Computing, 2022, 5 (3) : 259-268. doi: 10.3934/mfc.2021034

[10]

Natalia Tokareva. On the number of bent functions from iterative constructions: lower bounds and hypotheses. Advances in Mathematics of Communications, 2011, 5 (4) : 609-621. doi: 10.3934/amc.2011.5.609

[11]

Vaughn Climenhaga. A note on two approaches to the thermodynamic formalism. Discrete and Continuous Dynamical Systems, 2010, 27 (3) : 995-1005. doi: 10.3934/dcds.2010.27.995

[12]

J. Mead. $ \chi^2 $ test for total variation regularization parameter selection. Inverse Problems and Imaging, 2020, 14 (3) : 401-421. doi: 10.3934/ipi.2020019

[13]

Segismundo S. Izquierdo, Luis R. Izquierdo. "Test two, choose the better" leads to high cooperation in the Centipede game. Journal of Dynamics and Games, 2021  doi: 10.3934/jdg.2021018

[14]

Yu-Chi Chen. Security analysis of public key encryption with filtered equality test. Advances in Mathematics of Communications, 2021  doi: 10.3934/amc.2021053

[15]

Michael Jakobson, Lucia D. Simonelli. Countable Markov partitions suitable for thermodynamic formalism. Journal of Modern Dynamics, 2018, 13: 199-219. doi: 10.3934/jmd.2018018

[16]

Manfred Denker, Yuri Kifer, Manuel Stadlbauer. Thermodynamic formalism for random countable Markov shifts. Discrete and Continuous Dynamical Systems, 2008, 22 (1&2) : 131-164. doi: 10.3934/dcds.2008.22.131

[17]

Yongluo Cao, De-Jun Feng, Wen Huang. The thermodynamic formalism for sub-additive potentials. Discrete and Continuous Dynamical Systems, 2008, 20 (3) : 639-657. doi: 10.3934/dcds.2008.20.639

[18]

Anna Mummert. The thermodynamic formalism for almost-additive sequences. Discrete and Continuous Dynamical Systems, 2006, 16 (2) : 435-454. doi: 10.3934/dcds.2006.16.435

[19]

Manfred Denker, Yuri Kifer, Manuel Stadlbauer. Corrigendum to: Thermodynamic formalism for random countable Markov shifts. Discrete and Continuous Dynamical Systems, 2015, 35 (1) : 593-594. doi: 10.3934/dcds.2015.35.593

[20]

Yakov Pesin. On the work of Sarig on countable Markov chains and thermodynamic formalism. Journal of Modern Dynamics, 2014, 8 (1) : 1-14. doi: 10.3934/jmd.2014.8.1

 Impact Factor: 

Article outline

Figures and Tables

[Back to Top]