- Physical Review Journals
All JournalsPhysics Magazine
Access Provided by Xinjiang University

Physical Review E

Export citation

Export citation

Choose format for download:

Download Citation
  • Access by Xinjiang University

Statistics of correlations in nonlinear recurrent neural networks

Germán Mato, Facundo Rigatuso, and Gonzalo Torroba

PDF

Phys. Rev. E 114, 014226 – Published 31 July, 2026

DOI: https://doi.org/10.1103/lcfw-xrly

Abstract

The statistics of correlations are central quantities characterizing the collective dynamics of recurrent neural networks. We derive exact expressions for the statistics of correlations of nonlinear recurrent networks in the limit of a large number N of neurons, including systematic 1/N corrections, in the regime of Gaussian quenched disorder. Our approach uses a path-integral representation of the network's stochastic dynamics, which reduces the description to a few collective variables and enables efficient computation. This generalizes previous results on linear networks to include a wide family of nonlinear activation functions, which enter as interaction terms in the path integral. These interactions can resolve the instability of the linear theory and yield a strictly positive participation dimension. We present explicit results for power-law activations, revealing scaling behavior controlled by the network coupling. In addition, we introduce a class of activation functions based on Padé approximants and provide analytic predictions for their correlation statistics. Numerical simulations confirm our theoretical results with excellent agreement. We also compare with previous works that have studied the complementary case with annealed disorder, and based on this we propose a conjectural self-consistent equation for the more general case of colored noise.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (36)

  1. M. R. Cohen and A. Kohn, Measuring and interpreting neuronal correlations, Nat. Neurosci. 14, 811 (2011).
  2. I. Ginzburg and H. Sompolinsky, Theory of correlations in stochastic neural networks, Phys. Rev. E 50, 3171 (1994).
  3. L. F. Abbott and C. van Vreeswijk, Asynchronous states in networks of pulse-coupled oscillators, Phys. Rev. E 48, 1483 (1993).
  4. D. J. Amit and N. Brunel, Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex, Cereb. Cortex 7, 237 (1997).
  5. N. Brunel and V. Hakim, Fast global oscillations in networks of integrate-and-fire neurons with low firing rates, Neural Comput. 11, 1621 (1999).
  6. J. Hertz, Cross-correlations in high-conductance states of a model cortical network, Neural Comput. 22, 427 (2010).
  7. W. Bair, E. Zohary, and W. T. Newsome, Correlated firing in macaque visual area MT: Time scales and relationship to behavior, J. Neurosci. 21, 1676 (2001).
  8. C. Constantinidis and P. S. Goldman-Rakic, Correlated discharges among putative pyramidal neurons and interneurons in the primate prefrontal cortex, J. Neurophysiol. 88, 3487 (2002).
  9. E. Zohary, M. N. Shadlen, and W. T. Newsome, Correlated neuronal discharge rate and its implications for psychophysical performance, Nature (London) 370, 140 (1994).
  10. J. J. Pattadkal, R. T. O'Shea, D. Hansel, T. Taillefumier, D. H. Brager, and N. J. Priebe, Synchrony timescales underlie irregular neocortical spiking, Neuron 114, 724 (2026).
  11. S. Fusi, E. K. Miller, and M. Rigotti, Why neurons mix: High dimensionality for higher cognition, Curr. Opin. Neurobiol. 37, 66 (2016).
  12. P. Gao, E. Trautmann, B. Yu, G. Santhanam, S. Ryu, K. Shenoy, and S. Ganguli, A theory of multineuronal dimensionality, dynamics and measurement, bioRxiv (2017).
  13. D. Dahmen, S. Recanatesi, X. Jia, G. K. Ocker, L. Campagnola, S. Seeman, T. Jarsky, M. Helias, and E. Shea-Brown, Strong and localized recurrence controls dimensionality of neural activity across brain areas, bioRxiv (2020).
  14. D. Dahmen, S. Grün, M. Diesmann, and M. Helias, Second type of criticality in the brain uncovers rich multiple-neuron dynamics, Proc. Natl. Acad. Sci. USA 116, 13051 (2019).
  15. Y. Hu and H. Sompolinsky, The spectrum of covariance matrices of randomly connected recurrent neuronal networks with linear dynamics, PLoS Comput. Biol. 18, e1010327 (2022).
  16. D. G. Clark, L. F. Abbott, and A. Litwin-Kumar, Dimension of activity in random neural networks, Phys. Rev. Lett. 131, 118401 (2023).
  17. B. Zhang, D. J. Miller, and Y. Wang, Nonlinear system modeling with random matrices: Echo state networks revisited, IEEE Trans. Neural Networks Learn. Syst. 23, 175 (2011).
  18. X. Shen and Y. Hu, Covariance spectrum in nonlinear recurrent neural networks, arXiv:2508.05288.
  19. H. Sompolinsky, A. Crisanti, and H.-J. Sommers, Chaos in random neural networks, Phys. Rev. Lett. 61, 259 (1988).
  20. S. Song, P. J. Sjöström, M. Reigl, S. Nelson, and D. B. Chklovskii, Highly nonrandom features of synaptic connectivity in local cortical circuits, PLoS Biol. 3, e68 (2005).
  21. D. A. McCormick, B. W. Connors, J. W. Lighthall, and D. A. Prince, Comparative electrophysiology of pyramidal and sparsely spiny stellate neurons of the neocortex, J. Neurophys. 54, 782 (1985).
  22. M. Moshe and J. Zinn-Justin, Quantum field theory in the large n limit: A review, Phys. Rep. 385, 69 (2003).
  23. J. Schuecker, S. Goedeke, and M. Helias, Optimal sequence memory in driven random networks, Phys. Rev. X 8, 041029 (2018).
  24. O. Aharony, S. S. Gubser, J. M. Maldacena, H. Ooguri, and Y. Oz, Large n field theories, string theory and gravity, Phys. Rep. 323, 183 (2000).
  25. D. A. Roberts, S. Yaida, and B. Hanin, The Principles of Deep Learning Theory, Vol. 46 (Cambridge University Press, Cambridge, MA, 2022).
  26. M. Jazayeri and S. Ostojic, Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity, Curr. Opin. Neurobiol. 70, 113 (2021).
  27. I. Jolliffe, Principal component analysis, Encyclopedia of Statistics in Behavioral Science (J. Wiley & Sons, Hoboken, NJ, 2005).
  28. B. Ermentrout, Type I membranes, phase resetting curves, and synchrony, Neural Comput. 8, 979 (1996).
  29. B. S. Gutkin and G. B. Ermentrout, Dynamics of membrane excitability determine interspike interval variability: A link between spike generation mechanisms and cortical spike train statistics, Neural Comput. 10, 1047 (1998).
  30. E. M. Izhikevich, Dynamical Systems in Neuroscience (MIT Press, Cambridge, MA, 2007).
  31. A. Treves, Mean-field analysis of neuronal spike dynamics, Network: Comput. Neural Syst. 4, 259 (1993).
  32. Neural criticality repository (2026), https://github.com/german-mato/neural-criticality.
  33. G. A. Baker and P. Graves-Morris, Padé Approximants, 2nd ed., Encyclopedia of Mathematics and Its Applications, Vol. 59 (Cambridge University Press, Cambridge, UK, 1996).
  34. O. Keleş and A. M. Tekalp, Paon: A new neuron model using Padé approximants, in 2024 IEEE International Conference on Image Processing (ICIP) (IEEE, Piscataway, NJ, 2024), pp. 207–213.
  35. A. Molina, P. Schramowski, and K. Kersting, Padé activation units: End-to-end learning of flexible activation functions in deep networks, in 8th International Conference on Learning Representations (ICLR 2020).
  36. M. Layer, M. Helias, and D. Dahmen, Effect of synaptic heterogeneity on neuronal coordination, PRX Life 2, 013013 (2024).

Outline

Information

Phys. Rev. E 114, 014226– Published 31 July, 2026
  • Received 22 October 2025
  • Accepted 16 July 2026
Crossmark - Check for updates button

©2026 American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation