Article Contents
Article Contents

# Diagonal stationary points of the bethe functional

Both authors supported in part by Narodowe Centrum Nauki - grant 2015/17/B/ST1/00091.
• We investigate stationary points of the Bethe functional for the Ising model on a $2$-dimensional lattice. Such stationary points are also fixed points of message passing algorithms. In the absence of an external field, by symmetry reasons one expects the fixed points to have constant means at all sites. This is shown not to be the case. There is a critical value of the coupling parameter which is equal to the phase transition parameter on the computation tree, see [13], above which fixed points appear with means that are variable though constant on diagonals of the lattice and hence the term “diagonal stationary points”. A rigorous analytic proof of their existence is presented. Furthermore, computer-obtained examples of diagonal stationary points which are local maxima of the Bethe functional and hence stable equilibria for message passing are shown. The smallest such example was found on the $100× 100$ lattice.

Mathematics Subject Classification: Primary: 37N40, 90C26; Secondary: 37A60.

 Citation:

• Figure 1.  Illustration of the diagonal matrix which can be obtained from means given by Eq. (11).

Figure 2.  Numerical evidence that the means (from Eq. (11), visualized on Fig. 1) in fact define a stationary point of the Bethe functional. The dots on the graph show values of the negative Bethe functional computed for the means given by vector $B_{\eta}$ given by formula (12) with $\eta$ shown on the horizontal axis and various randomly chosen $(X_{\ell})$.

Figure 3.  Values of the negative Bethe functional for the diagonal stationary point $\mathcal{B}_0$ perturbed in the direction of $P$ according to formula (12).

Figure 5.  The stability test algorithm.

Figure 4.  Stable fixed point given by Eq. (13).

•  [1] R. J. Baxter, Exactly solved models in statistical mechanics, Integrable Systems in Statistical Mechanics, 1 (1985), 5-63.  doi: 10.1142/9789814415255_0002. [2] H. A. Bethe, Statistical theory of superlattices, Selected Works of Hans A Bethe, 18 (1997), 245-270.  doi: 10.1142/9789812795755_0010. [3] S. Dorogovtsev, A. Goltsev and J. Mendes, Critical phenomena in complex networks, Rev. Mod. Phys., 80 (2008), 1275-1335.  doi: 10.1103/RevModPhys.80.1275. [4] C. Fortuin, P. Kasteleyn and J. Ginibre, Correlation inequalities on some partially ordered sets, Commun. Math. Phys., 22 (1971), 89-103.  doi: 10.1007/BF01651330. [5] T. Heskes, Stable fixed points of loopy belief propagation are local minima of the Bethe free energy, In S. Thrun S. Becker and K. Obermayer, editors, Advances in Neural Information Processing Systems 15, MIT Press, Cambridge, MA, (2003), 343-350. [6] J. M. Mooij and H. J. Kappen, On the properties of the Bethe approximation and loopy belief propagation on binary networks J. Stat. Mech. Theor. Exp. 11 (2005), P11012. doi: 10.1088/1742-5468/2005/11/P11012. [7] J. M. Mooij and H. J. Kappen, Sufficient conditions for convergence of the sum-product algorithm, IEEE Transactions on Information Theory, 53 (2007), 4422-4437.  doi: 10.1109/TIT.2007.909166. [8] S. Newhouse, Diffeomorphisms with infinitely many sinks, Topology, 13 (1974), 9-18.  doi: 10.1016/0040-9383(74)90034-2. [9] J. Pearl, Reverend Bayes on inference engines: A distributed hierarchical approach, Proceedings of the Second National Conference on Artificial Intelligence, (1982), 133-136. [10] T. G. Roosta and M. J. Wainwright snd S. S. Sastry, Convergence analysis of reweighted sum-product algorithms, IEEE Transactions on Signal Processing, 56 (2008), 4293-4305.  doi: 10.1109/ICASSP.2007.366292. [11] J. Shin, The complexity of approximating a bethe equilibrium, IEEE Transactions on Information Theory, 60 (2014), 3959-3969.  doi: 10.1109/TIT.2014.2317487. [12] G. Siudem and G. Świątek, Dynamics of the belief propagation for the ising model, Acta Physica Polonica A, 127 (2015), 3A145-3A149.  doi: 10.12693/APhysPolA.127.A-145. [13] S. Tatikonda and M. Jordan, Loopy belief propagation and Gibbs measures, in Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann, San Francisco, (2002), 493-500, [14] M. J. Wainwright and M. I. Jordan, Graphical models, exponential families, and variational inference, Foundations and Trends in Machine Learning, 1 (2008), 1-305.  doi: 10.1561/2200000001. [15] A. Weller and T. Jebara, Bethe bounds and approximating the global optimum, Journal of Machine Learning Research W& CP, 31 (2013), 618-631. [16] M. Welling and Y. -W. Teh, Belief Optimization for Binary Networks: A Stable Alternative to Loopy Belief Propagation in Proc. 17th Conference on Uncertainty in Artificial Intelligence (UAI), 2001. [17] J. Yedidia, W. Freeman and Y. Weiss, Constructing free-energy approximations and generalized belief propagation algorithms, IEEE Trans. on Information Theory, 51 (2005), 2282-2312.  doi: 10.1109/TIT.2005.850085.

Figures(5)