# American Institute of Mathematical Sciences

November  2020, 3(4): 301-308. doi: 10.3934/mfc.2020012

## AIMS: Average information matrix splitting

 1 Laboratory for Intelligent Computing and Financial Technology, Department of Mathematics, Xi'an Jiaotong-Liverpool University, Suzhou, 215123, China 2 Laboratory of Computational Physics, Institute of Applied Physics and Computational Mathematics, Beijing 100088, China

* Corresponding author: Shengxin Zhu

Received  November 2019 Revised  February 2020 Published  June 2020

Fund Project: This research is supported by Foundation of LCP(6142A05180501), Jiangsu Science and Technology Basic Research Program (BK20171237), Key Program Special Fund of XJTLU (KSF-E-21, KSF-P-02), Research Development Fund of XJTLU (RDF-2017-02-23) and partially supported by NSFC (No.11771002, 11571047, 11671049, 11671051, 6162003, and 11871339)

For linear mixed models with co-variance matrices which are not linearly dependent on variance component parameters, we prove that the average of the observed information and the Fisher information can be split into two parts. The essential part enjoys a simple and computational friendly formula, while the other part which involves a lot of computations is a random zero matrix and thus is negligible.

Citation: Shengxin Zhu, Tongxiang Gu, Xingping Liu. AIMS: Average information matrix splitting. Mathematical Foundations of Computing, 2020, 3 (4) : 301-308. doi: 10.3934/mfc.2020012
##### References:
 [1] Z. Chen, S. Zhu, Q. Niu and X. Lu, Censorious young: Knowledge discovery from high-throughput movie rating data with LME4, in 2019 IEEE 4th International Conference on Big Data Analytics (ICBDA), 2019, 32–36. doi: 10.1109/ICBDA.2019.8713193.  Google Scholar [2] Z. Chen, S. Zhu, Q. Niu and T. Zuo, Knowledge discovery and recommendation with linear mixed model, IEEE Access, 8 (2020), 38304-38317.  doi: 10.1109/ACCESS.2020.2973170.  Google Scholar [3] B. Efron and D. V. Hinkley, Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information, Biometrika, 65 (1978), 457-483.  doi: 10.1093/biomet/65.3.457.  Google Scholar [4] R. A. Fisher, The Genetical Theory of Natural Selection, Oxford University Press, Oxford, 1999.  Google Scholar [5] B. Gao, G. Zhan, H. Wang, Y. Wang and S. Zhu, Learning with linear mixed model for group recommendation systems, in Proceedings of the 2019 11th International Conference on Machine Learning and Computing, ICMLC '19, Association for Computing Machinery, New York, NY, 2019, 81–85. doi: 10.1145/3318299.3318342.  Google Scholar [6] A. R. Gilmour, R. Thompson and B. R. Cullis, Average information reml: An efficient algorithm for variance parameter estimation in linear mixed models, Biometrics, 51 (1995), 1440-1450.  doi: 10.2307/2533274.  Google Scholar [7] G. Givens and J. Hoeting, Computational Statistics, 2$^{nd}$ edition, Wiley Series in Computation Statistics, John Wiley & Sons, Inc., Wiley, NJ, 2005.  Google Scholar [8] F. N. Gumedze and T. T. Dunne, Parameter estimation and inference in the linear mixed model, Linear Algebra Appl., 435 (2011), 1920-1944.  doi: 10.1016/j.laa.2011.04.015.  Google Scholar [9] A. Heavens, Generalised Fisher matrices, Entropy, 18 (2016), 8 pp. doi: 10.3390/e18060236.  Google Scholar [10] W. Janke, D. Johnston and R. Kenna, Information geometry and phase transitions, Physica A: Statistical Mechanics and its Applications, 336 (2004), 181-186.  doi: 10.1016/j.physa.2004.01.023.  Google Scholar [11] R. I. Jennrich and P. F. Sampson, Newton-Raphson and related algorithms for maximum likelihood variance component estimation, Technometrics, 18 (1976), 11-17.  doi: 10.2307/1267911.  Google Scholar [12] D. Johnson and R. Thompson, Restricted maximum likelihood estimation of variance components for univariate animal models using sparse matrix techniques and average information, Journal of Dairy Science, 78 (1995), 449-456.  doi: 10.3168/jds.S0022-0302(95)76654-1.  Google Scholar [13] N. T. Longford, A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects, Biometrika, 74 (1987), 817-827.  doi: 10.1093/biomet/74.4.817.  Google Scholar [14] K. Meyer, An average information restricted maximum likelihood algorithm for estimating reduced rank genetic covariance matrices or covariance functions for animal models with equal design matrices, Genetics Selection Evolution, 29 (1997), 97. doi: 10.1186/1297-9686-29-2-97.  Google Scholar [15] J. I. Myung and D. J. Navarro, Information Matrix, American Cancer Society, 2005. doi: 10.1002/0470013192.bsa302.  Google Scholar [16] J. W. Pratt, F. Y. Edgeworth and R. A. Fisher on the efficiency of maximum likelihood estimation, Ann. Statist., 4 (1976), 501-514.  doi: 10.1214/aos/1176343457.  Google Scholar [17] M. Prokopenko, J. T. Lizier, O. Obst and X. R. Wang, Relating Fisher information to order parameters, Phys. Rev. E, 84 (2011), 041116. doi: 10.1103/PhysRevE.84.041116.  Google Scholar [18] S. R. Searle, G. Casella and C. E. McCulloch, Variance Components, Wiley Series in Probability and Statistics, Wiley-Interscience [John Wiley & Sons], Hoboken, NJ, 2006.  Google Scholar [19] M. Vallisneri, A User Manual for the Fisher Informaiton Matrix, California Institute of Technology, Jet Propulsion Laboratory, 2007. Google Scholar [20] R. S. Varga, Matrix Iterative Analysis, expanded edition, Springer Series in Computational Mathematics, 27, Springer-Verlag, Berlin, 2000. doi: 10.1007/978-3-642-05156-2.  Google Scholar [21] Y. Wang, T. Wu, F. Ma and S. Zhu, Personalized recommender systems with multiple source data, in Computing Conference 2020 Google Scholar [22] S. Welham, S. Zhu and A. J. Wathen, Big Data, Fast Models: Faster Calculation of Models from High-Throughput Biological Data Sets, Knowledge Transfer Report IP12-0009, Smith Institute and The Universtiy of Oxford, Oxford, 2013. Google Scholar [23] R. Zamir, A Necessary and Sufficient Condition for Equality in the Matrix Fisher Information Inequality, Technical report, Tel Aviv University, 1997.  Google Scholar [24] R. Zamir, A proof of the Fisher information inequality via a data processing argument, IEEE Transactions on Information Theory, 44 (1998), 1246-1250.  doi: 10.1109/18.669301.  Google Scholar [25] S. Zhu, T. Gu and X. Liu, Information matrix splitting, preprint, arXiv: 1605.07646. Google Scholar [26] S. Zhu and A. J. Wathen, Essential formulae for restricted maximum likelihood and its derivatives associated with the linear mixed models, preprint, arXiv: 1805.05188. Google Scholar [27] S. Zhu and A. J. Wathen, Sparse inversion for derivative of log determinant, arXiv: 1911.00685. Google Scholar [28] T. Zuo, S. Zhu and J. Lu, A hybrid recommender system combing singular value decomposition and linear mixed model, in Computing Conference 2020, Advance in Intelligent Systems and Computing, Springer International Publishing, Cham, 2020. Google Scholar

show all references

##### References:
 [1] Z. Chen, S. Zhu, Q. Niu and X. Lu, Censorious young: Knowledge discovery from high-throughput movie rating data with LME4, in 2019 IEEE 4th International Conference on Big Data Analytics (ICBDA), 2019, 32–36. doi: 10.1109/ICBDA.2019.8713193.  Google Scholar [2] Z. Chen, S. Zhu, Q. Niu and T. Zuo, Knowledge discovery and recommendation with linear mixed model, IEEE Access, 8 (2020), 38304-38317.  doi: 10.1109/ACCESS.2020.2973170.  Google Scholar [3] B. Efron and D. V. Hinkley, Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information, Biometrika, 65 (1978), 457-483.  doi: 10.1093/biomet/65.3.457.  Google Scholar [4] R. A. Fisher, The Genetical Theory of Natural Selection, Oxford University Press, Oxford, 1999.  Google Scholar [5] B. Gao, G. Zhan, H. Wang, Y. Wang and S. Zhu, Learning with linear mixed model for group recommendation systems, in Proceedings of the 2019 11th International Conference on Machine Learning and Computing, ICMLC '19, Association for Computing Machinery, New York, NY, 2019, 81–85. doi: 10.1145/3318299.3318342.  Google Scholar [6] A. R. Gilmour, R. Thompson and B. R. Cullis, Average information reml: An efficient algorithm for variance parameter estimation in linear mixed models, Biometrics, 51 (1995), 1440-1450.  doi: 10.2307/2533274.  Google Scholar [7] G. Givens and J. Hoeting, Computational Statistics, 2$^{nd}$ edition, Wiley Series in Computation Statistics, John Wiley & Sons, Inc., Wiley, NJ, 2005.  Google Scholar [8] F. N. Gumedze and T. T. Dunne, Parameter estimation and inference in the linear mixed model, Linear Algebra Appl., 435 (2011), 1920-1944.  doi: 10.1016/j.laa.2011.04.015.  Google Scholar [9] A. Heavens, Generalised Fisher matrices, Entropy, 18 (2016), 8 pp. doi: 10.3390/e18060236.  Google Scholar [10] W. Janke, D. Johnston and R. Kenna, Information geometry and phase transitions, Physica A: Statistical Mechanics and its Applications, 336 (2004), 181-186.  doi: 10.1016/j.physa.2004.01.023.  Google Scholar [11] R. I. Jennrich and P. F. Sampson, Newton-Raphson and related algorithms for maximum likelihood variance component estimation, Technometrics, 18 (1976), 11-17.  doi: 10.2307/1267911.  Google Scholar [12] D. Johnson and R. Thompson, Restricted maximum likelihood estimation of variance components for univariate animal models using sparse matrix techniques and average information, Journal of Dairy Science, 78 (1995), 449-456.  doi: 10.3168/jds.S0022-0302(95)76654-1.  Google Scholar [13] N. T. Longford, A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects, Biometrika, 74 (1987), 817-827.  doi: 10.1093/biomet/74.4.817.  Google Scholar [14] K. Meyer, An average information restricted maximum likelihood algorithm for estimating reduced rank genetic covariance matrices or covariance functions for animal models with equal design matrices, Genetics Selection Evolution, 29 (1997), 97. doi: 10.1186/1297-9686-29-2-97.  Google Scholar [15] J. I. Myung and D. J. Navarro, Information Matrix, American Cancer Society, 2005. doi: 10.1002/0470013192.bsa302.  Google Scholar [16] J. W. Pratt, F. Y. Edgeworth and R. A. Fisher on the efficiency of maximum likelihood estimation, Ann. Statist., 4 (1976), 501-514.  doi: 10.1214/aos/1176343457.  Google Scholar [17] M. Prokopenko, J. T. Lizier, O. Obst and X. R. Wang, Relating Fisher information to order parameters, Phys. Rev. E, 84 (2011), 041116. doi: 10.1103/PhysRevE.84.041116.  Google Scholar [18] S. R. Searle, G. Casella and C. E. McCulloch, Variance Components, Wiley Series in Probability and Statistics, Wiley-Interscience [John Wiley & Sons], Hoboken, NJ, 2006.  Google Scholar [19] M. Vallisneri, A User Manual for the Fisher Informaiton Matrix, California Institute of Technology, Jet Propulsion Laboratory, 2007. Google Scholar [20] R. S. Varga, Matrix Iterative Analysis, expanded edition, Springer Series in Computational Mathematics, 27, Springer-Verlag, Berlin, 2000. doi: 10.1007/978-3-642-05156-2.  Google Scholar [21] Y. Wang, T. Wu, F. Ma and S. Zhu, Personalized recommender systems with multiple source data, in Computing Conference 2020 Google Scholar [22] S. Welham, S. Zhu and A. J. Wathen, Big Data, Fast Models: Faster Calculation of Models from High-Throughput Biological Data Sets, Knowledge Transfer Report IP12-0009, Smith Institute and The Universtiy of Oxford, Oxford, 2013. Google Scholar [23] R. Zamir, A Necessary and Sufficient Condition for Equality in the Matrix Fisher Information Inequality, Technical report, Tel Aviv University, 1997.  Google Scholar [24] R. Zamir, A proof of the Fisher information inequality via a data processing argument, IEEE Transactions on Information Theory, 44 (1998), 1246-1250.  doi: 10.1109/18.669301.  Google Scholar [25] S. Zhu, T. Gu and X. Liu, Information matrix splitting, preprint, arXiv: 1605.07646. Google Scholar [26] S. Zhu and A. J. Wathen, Essential formulae for restricted maximum likelihood and its derivatives associated with the linear mixed models, preprint, arXiv: 1805.05188. Google Scholar [27] S. Zhu and A. J. Wathen, Sparse inversion for derivative of log determinant, arXiv: 1911.00685. Google Scholar [28] T. Zuo, S. Zhu and J. Lu, A hybrid recommender system combing singular value decomposition and linear mixed model, in Computing Conference 2020, Advance in Intelligent Systems and Computing, Springer International Publishing, Cham, 2020. Google Scholar
 [1] Nicolas Rougerie. On two properties of the Fisher information. Kinetic & Related Models, 2021, 14 (1) : 77-88. doi: 10.3934/krm.2020049 [2] Wai-Ki Ching, Jia-Wen Gu, Harry Zheng. On correlated defaults and incomplete information. Journal of Industrial & Management Optimization, 2021, 17 (2) : 889-908. doi: 10.3934/jimo.2020003 [3] Hui Gao, Jian Lv, Xiaoliang Wang, Liping Pang. An alternating linearization bundle method for a class of nonconvex optimization problem with inexact information. Journal of Industrial & Management Optimization, 2021, 17 (2) : 805-825. doi: 10.3934/jimo.2019135 [4] Honglin Yang, Jiawu Peng. Coordinating a supply chain with demand information updating. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020181 [5] Yueyang Zheng, Jingtao Shi. A stackelberg game of backward stochastic differential equations with partial information. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020047 [6] Chuan Ding, Da-Hai Li. Angel capitalists exit decisions under information asymmetry: IPO or acquisitions. Journal of Industrial & Management Optimization, 2021, 17 (1) : 369-392. doi: 10.3934/jimo.2019116 [7] Wenqin Zhang, Zhengchun Zhou, Udaya Parampalli, Vladimir Sidorenko. Capacity-achieving private information retrieval scheme with a smaller sub-packetization. Advances in Mathematics of Communications, 2021, 15 (2) : 347-363. doi: 10.3934/amc.2020070 [8] Zhongbao Zhou, Yanfei Bai, Helu Xiao, Xu Chen. A non-zero-sum reinsurance-investment game with delay and asymmetric information. Journal of Industrial & Management Optimization, 2021, 17 (2) : 909-936. doi: 10.3934/jimo.2020004 [9] Editorial Office. Retraction: Xiaohong Zhu, Lihe Zhou, Zili Yang and Joyati Debnath, A new text information extraction algorithm of video image under multimedia environment. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1265-1265. doi: 10.3934/dcdss.2019087 [10] Nalin Fonseka, Jerome Goddard II, Ratnasingham Shivaji, Byungjae Son. A diffusive weak Allee effect model with U-shaped emigration and matrix hostility. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020356 [11] Peizhao Yu, Guoshan Zhang, Yi Zhang. Decoupling of cubic polynomial matrix systems. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 13-26. doi: 10.3934/naco.2020012 [12] Chongyang Liu, Meijia Han, Zhaohua Gong, Kok Lay Teo. Robust parameter estimation for constrained time-delay systems with inexact measurements. Journal of Industrial & Management Optimization, 2021, 17 (1) : 317-337. doi: 10.3934/jimo.2019113 [13] Dominique Chapelle, Philippe Moireau, Patrick Le Tallec. Robust filtering for joint state-parameter estimation in distributed mechanical systems. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 65-84. doi: 10.3934/dcds.2009.23.65 [14] Parikshit Upadhyaya, Elias Jarlebring, Emanuel H. Rubensson. A density matrix approach to the convergence of the self-consistent field iteration. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 99-115. doi: 10.3934/naco.2020018 [15] S. Sadeghi, H. Jafari, S. Nemati. Solving fractional Advection-diffusion equation using Genocchi operational matrix based on Atangana-Baleanu derivative. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020435 [16] Yuri Fedorov, Božidar Jovanović. Continuous and discrete Neumann systems on Stiefel varieties as matrix generalizations of the Jacobi–Mumford systems. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020375 [17] Sihem Guerarra. Maximum and minimum ranks and inertias of the Hermitian parts of the least rank solution of the matrix equation AXB = C. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 75-86. doi: 10.3934/naco.2020016 [18] Dan Zhu, Rosemary A. Renaut, Hongwei Li, Tianyou Liu. Fast non-convex low-rank matrix decomposition for separation of potential field data using minimal memory. Inverse Problems & Imaging, 2021, 15 (1) : 159-183. doi: 10.3934/ipi.2020076 [19] Jiannan Zhang, Ping Chen, Zhuo Jin, Shuanming Li. Open-loop equilibrium strategy for mean-variance portfolio selection: A log-return model. Journal of Industrial & Management Optimization, 2021, 17 (2) : 765-777. doi: 10.3934/jimo.2019133 [20] Ying Liu, Yanping Chen, Yunqing Huang, Yang Wang. Two-grid method for semiconductor device problem by mixed finite element method and characteristics finite element method. Electronic Research Archive, 2021, 29 (1) : 1859-1880. doi: 10.3934/era.2020095

Impact Factor: