- Physical Review Journals
All JournalsPhysics Magazine
Access Provided by Xinjiang University

Physical Review E

Export citation

Export citation

Choose format for download:

Download Citation
  • Access by Xinjiang University

Principal-component-analysis eigenvalue spectra from data with symmetry-breaking structure

D. C. Hoyle* and M. Rattray†

  • Department of Computer Science, University of Manchester, Kilburn Building, Oxford Road, Manchester M13 9PL, United Kingdom

  • *Electronic address: david.c.hoyle@man.ac.uk; www.cs.man.ac.uk/∼dchoyle
  • †Electronic address: magnus@cs.man.ac.uk www.cs.man.ac.uk/∼magnus
PDF

Phys. Rev. E 69, 026124 – Published 27 February, 2004

DOI: https://doi.org/10.1103/PhysRevE.69.026124

Abstract

Principal component analysis (PCA) is a ubiquitous method of multivariate statistics that focuses on the eigenvalues λ and eigenvectors of the sample covariance matrix of a data set. We consider p, N-dimensional data vectors ξ drawn from a distribution with covariance matrix C. We use the replica method to evaluate the expected eigenvalue distribution ρ(λ) as N→∞ with p=αN for some fixed α. In contrast to existing studies we consider the case where C contains a number of symmetry-breaking directions, so that the sample data set contains some definite structure. Explicitly we set C=σ2I+σ2∑m=1SAmBmBmT, with Am>0∀m. We find that the bulk of the eigenvalues are distributed as for the case when the elements of ξ are independent and identically distributed. With increasing α a series of phase transitions are observed, at α=Am−2,m=1,2,…,S, each time a single δ function, δ(λ−λu(Am)), separates from the upper edge of the bulk distribution, where λu(A)=σ2[1+A][1+(αA)−1]. We confirm the results of the replica analysis by studying the Stieltjes transform of ρ(λ). This suggests that the results obtained from the replica analysis are universal, irrespective of the distribution from which ξ is drawn, provided the fourth moment of each element of ξ exists.

View figure in article

References (43)

  1. A. Engel and C. Van den Broeck, Statistical Mechanics of Learning (Cambridge University Press, Cambridge, 2001).
  2. I. T. Jolliffe, Principal Component Analysis (Springer-Verlag, New York, 1986).
  3. M.E. Tipping and C. Bishop, J. R. Stat. Soc. Ser. B. Methodol. 61, 611 (1999).
  4. M.E. Tipping and C. Bishop, Neural Comput. 11, 443 (1999).
  5. K. W. Wachter, in Proceedings of the Ninth Interface Symposium Computer Science and Statistics, edited by David C. Hoaglin and Roy E. Welsch (Prindle, Weber and Schmidt, Boston, 1976), p. 299.
  6. T.W. Anderson, Ann. Math. Stat. 34, 122 (1963).
  7. I.M. Johnstone, Ann. Stat. 29, 295 (2001).
  8. A. Soshnikov, J. Stat. Phys. 108, 1033 (2002).
  9. J. Wishart, Biometrika 20, 32 (1928).
  10. R.A. Janik and M.A. Nowak, J. Phys. A 36, 3629 (2003).
  11. M. L. Mehta, Random Matrices (Academic Press, San Diego, 1991).
  12. Z.D. Bai, Statistica Sinica 9, 611 (1999).
  13. P.J. Forrester, N.C. Snaith, and J.J.M. Verbaarschot, J. Phys. A 36, R1 (2003).
  14. V.A. Marčenko and L.A. Pastur, Math. USSR. Sb. 1, 507 (1967).
  15. A. Edelman, SIAM J. Matrix Anal. Appl. 9, 543 (1988).
  16. K.W. Wachter, Ann. Probab. 6, 1 (1978).
  17. C.A. Tracy and H. Widom, Commun. Math. Phys. 177, 727 (1996).
  18. P. Reimann, C. Van den Broeck, and G.J. Bex, J. Phys. A 29, 3521 (1996).
  19. P. Reimann and C. Van den Broeck, Phys. Rev. E 53, 3989 (1996).
  20. S.F. Edwards and R.C. Jones, J. Phys. A 9, 1595 (1976).
  21. G.J. Rodgers and A.J. Bray, Phys. Rev. B 37, 3557 (1988).
  22. H.J. Sommers, A. Crisanti, H. Sompolinsky, and Y. Stein, Phys. Rev. Lett. 60, 1895 (1988).
  23. A.M. Sengupta and P.P. Mitra, Phys. Rev. E 60, 3389 (1999).
  24. G. M. Cicuta, in Random Matrices and Their Applications, edited by P. M. Bleher and A. R. Its, Mathematical Sciences Research Institute Publications Vol. 40 (Cambridge University Press, Cambridge, 2001), p. 95.
  25. I. Derényi, T. Geszti, and G. Györgyi, Phys. Rev. E 50, 3192 (1994).
  26. D.C. Hoyle and M. Rattray, Europhys. Lett. 62, 117 (2003).
  27. P. Sollich, J. Phys. A 27, 7771 (1994).
  28. P. Sollich, in Advances in Neural Information Processing Systems 7, edited by G. Tesauro, D. S. Touretzky, and T. K. Leen (MIT Press, Cambridge, MA 1995), p. 207.
  29. G.S. Dhesi and R.C. Jones, J. Phys. A 23, 5577 (1990).
  30. J.J.M. Verbaarschot and M.R. Zirnbauer, Ann. Phys. (N.Y.) 158, 78 (1984).
  31. M. Opper, Europhys. Lett. 8, 389 (1989).
  32. N.H. Kuiper, Proc. K. Ned. Akad. Wet., Ser. A: Math. Sci. 63, 38 (1962).
  33. M.A. Stephens, J. R. Stat. Soc. Ser. B. Methodol. 32, 115 (1970).
  34. D. C. Hoyle and M. Rattray (unpublished).
  35. J.W. Silverstein and S. Choi, J. Multivariate Anal. 54, 295 (1995).
  36. J.W. Silverstein and P.L. Combettes, IEEE Trans. Signal Process. 40, 2100 (1992).
  37. Z.D. Bai, Ann. Probab. 21, 649 (1993).
  38. P.R. Rider, Ann. Inst. Stat. Math. 9, 215 (1957).
  39. J.H. Miller and J.B. Thomas, IEEE Trans. Inf. Theory 18, 241 (1972).
  40. Z.D. Bai and J.W. Silverstein, Ann. Probab. 26, 316 (1998).
  41. Z.D. Bai and J.W. Silverstein, Ann. Probab. 27, 1536 (1999).
  42. Y. Le Cun, I. Kanter, and S.A. Solla, Phys. Rev. Lett. 66, 2396 (1991).
  43. S. Halkjær and O. Winther, in Neural Information Processing Systems 9, edited by M. Mozer, M. Jordan, and T. Petsche (MIT Press, Cambridge, MA 1997), p. 169.

Outline

Information

Phys. Rev. E 69, 026124– Published 27 February, 2004
  • Received 2 April 2003

© 2004 American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation