- Physical Review Journals
All JournalsPhysics Magazine
Access Provided by Xinjiang University

Physical Review E

Export citation

Export citation

Choose format for download:

Download Citation
  • Editors' Suggestion
  • Access by Xinjiang University

Low-resolution descriptions of model neural activity reveal hidden features and underlying system properties

Riccardo Aldrigo1, Roberto Menichetti1,2, and Raffaello Potestio1,2,*

  • *Contact author: raffaello.potestio@unitn.it
PDF

Phys. Rev. E 111, 044315 – Published 21 April, 2025

DOI: https://doi.org/10.1103/PhysRevE.111.044315

Abstract

The analysis of complex systems such as neural networks is made particularly difficult by the overwhelming number of their interacting components. In the absence of prior knowledge, identifying a small but informative subset of network nodes on which the analysis should focus is a rather challenging task. In this work, we address this problem in the context of a Hopfield model, which is observed through the lenses of low-resolution representations, or decimation mappings, consisting of subgroups of its neurons. The optimal, most informative mappings of the network are defined through a recently developed methodology, the mapping entropy optimization workflow (MEOW), which performs an unsupervised analysis of the states sampled by the network and identifies those subgroups of units whose configuration distribution is closest to that of the full, high-resolution model. Which neurons are retained in an optimal mapping is found to critically depend on the properties of the interaction matrix of the network and the level of detail employed to describe the system; by these means, it is thus possible to extract quantitative insight about the underlying properties of the high-resolution model through the analysis of its optimal low-resolution representations. These results show a tight and potentially fruitful relation between the level of detail at which the network is inspected and the type and amount of information that can be gathered from it, and showcase the MEOW approach as a practical, enabling tool for the study of complex systems.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (56)

  1. E. Rolls, A. Treves et al., Neural Networks and Brain Function (Oxford University Press, Oxford, UK, 1998).
  2. P. Dayan and L. F. Abbott, Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems (MIT press, Cambridge, London, 2005).
  3. G. Zamora-López, E. Russo, P. M. Gleiser, C. Zhou, and J. Kurths, Characterizing the complexity of brain and mind networks, Philos. Trans. R. Soc. A 369, 3730 (2011).
  4. C. W. Lynn and D. S. Bassett, The physics of brain network structure, function and control, Nat. Rev. Phys. 1, 318 (2019).
  5. S. Herculano-Houzel, The human brain in numbers: A linearly scaled-up primate brain, Front. Human Neurosci. 3, 31 (2009).
  6. A. Joudaki, N. Salehi, M. Jalili, and M. G. Knyazeva, EEG-based functional brain networks: Does the network size matter? PLoS ONE 7, e35673 (2012).
  7. E. Kropff and A. Treves, The storage capacity of Potts models for semantic memory retrieval, J. Stat. Mech. (2005) P08010.
  8. E. Russo, V. M. Namboodiri, A. Treves, and E. Kropff, Free association transitions in models of cortical latching dynamics, New J. Phys. 10, 015008 (2008).
  9. D. H. Ackley, G. E. Hinton, and T. J. Sejnowski, A learning algorithm for Boltzmann machines, Cognit. Sci. 9, 147 (1985).
  10. E. Aarts and J. Korst, Simulated Annealing and Boltzmann Machines: A Stochastic Approach to Combinatorial Optimization and Neural Computing (John Wiley & Sons, Inc., New York, NY, 1989).
  11. C. Marullo and E. Agliari, Boltzmann machines as generalized Hopfield networks: A review of recent results and outlooks, Entropy 23, 34 (2020).
  12. S. Goel, A. Klivans, and F. Koehler, From Boltzmann machines to neural networks and back again, Adv. Neural Inf. Proces. Syst. 33, 6354 (2020).
  13. O. Sporns, Network neuroscience, The Future of the Brain: Essays by the World's Leading Neuroscientists (Princeton University Press, Princeton, 2014), p. 90.
  14. D. S. Bassett, P. Zurn, and J. I. Gold, On the nature and use of models in network neuroscience, Nat. Rev. Neurosci. 19, 566 (2018).
  15. T. P. Vogels, K. Rajan, and L. Abbott, Neural network dynamics, Annu. Rev. Neurosci. 28, 357 (2005).
  16. J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. Natl. Acad. Sci. USA 79, 2554 (1982).
  17. J. J. Hopfield, Neurons with graded response have collective computational properties like those of two-state neurons, Proc. Natl. Acad. Sci. USA 81, 3088 (1984).
  18. A. D. Bruce, E. J. Gardner, and D. J. Wallace, Dynamics and statistical mechanics of the Hopfield model, J. Phys. A: Math. Gen. 20, 2909 (1987).
  19. H. Ramsauer, B. Schäfl, J. Lehner, P. Seidl, M. Widrich, T. Adler, L. Gruber, M. Holzleitner, M. Pavlović, G. K. Sandve et al., Hopfield networks is all you need, arXiv:2008.02217.
  20. A. Barra, G. Genovese, and F. Guerra, The replica symmetric approximation of the analogical neural network, J. Stat. Phys. 140, 784 (2010).
  21. E. Agliari, D. Migliozzi, and D. Tantari, Non-convex multi-species Hopfield models, J. Stat. Phys. 172, 1247 (2018).
  22. E. Agliari, L. Albanese, A. Barra, and G. Ottaviani, Replica symmetry breaking in neural networks: A few steps toward rigorous results, J. Phys. A: Math. Theor. 53, 415005 (2020).
  23. F. Alemanno, L. Camanzi, G. Manzan, and D. Tantari, Hopfield model with planted patterns: A teacher-student self-supervised learning model, Appl. Math. Comput. 458, 128253 (2023).
  24. A. Crisanti, D. J. Amit, and H. Gutfreund, Saturation level of the Hopfield model for neural network, Europhys. Lett. 2, 337 (1986).
  25. J. Bruck, On the convergence properties of the Hopfield model, Proc. IEEE 78, 1579 (1990).
  26. H. Huang, Statistical Mechanics of Neural Networks (Springer, Berlin, 2021).
  27. M. Giulini, R. Menichetti, M. S. Shell, and R. Potestio, An information-theory-based approach for optimal model reduction of biomolecules, J. Chem. Theory Comput. 16, 6795 (2020).
  28. F. Errica, M. Giulini, D. Bacciu, R. Menichetti, A. Micheli, and R. Potestio, A deep graph network–enhanced sampling approach to efficiently explore the space of reduced representations of proteins, Front. Mol. Biosci. 8, 637396 (2021).
  29. R. Holtzman, M. Giulini, and R. Potestio, Making sense of complex systems through resolution, relevance, and mapping entropy, Phys. Rev. E 106, 044101 (2022).
  30. M. Giulini, R. Fiorentini, L. Tubiana, R. Potestio, and R. Menichetti, Excogito, an extensible coarse-graining toolbox for the investigation of biomolecules by means of low-resolution representation, J. Chem. Inf. Model. 64, 4912 (2024).
  31. M. Rigoli, R. Potestio, and R. Menichetti, A multiscale analysis of the CzrA transcription repressor highlights the allosteric changes induced by metal ion binding, J. Phys. Chem. B 129, 611 (2025).
  32. S. Kullback and R. A. Leibler, On information and sufficiency, Ann. Math. Stat. 22, 79 (1951).
  33. M. S. Shell, The relative entropy is fundamental to multiscale and inverse thermodynamic problems, J. Chem. Phys. 129, 14 (2008).
  34. J. F. Rudzinski and W. Noid, Coarse-graining entropy, forces, and structures, J. Chem. Phys. 135, 21 (2011).
  35. W. Noid, Perspective: Advances, challenges, and insight for predictive coarse-grained models, J. Phys. Chem. B 127, 4174 (2023).
  36. W. McCulloch and W. Pitts, A logical calculus of the ideas immanent in nervous activity, Bull. Math. Biophys. 5, 115 (1943).
  37. F. Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms (Spartan Books, Washington, DC, 1962).
  38. M. Minsky and S. Papert, Perceptrons: An Introduction to Computational Geometry (MIT Press, Cambridge, MA, 1969).
  39. D. J. Amit, H. Gutfreund, and H. Sompolinsky, Storing infinite numbers of patterns in a spin-glass model of neural networks, Phys. Rev. Lett. 55, 1530 (1985).
  40. D. J. Amit, Modeling Brain Function: The World of Attractor Neural Networks (Cambridge University Press, Cambridge, UK, 1989).
  41. R. J. Glauber, Time-dependent statistics of the Ising model, J. Math. Phys. 4, 294 (1963).
  42. D. J. Amit, H. Gutfreund, and H. Sompolinsky, Spin-glass models of neural networks, Phys. Rev. A 32, 1007 (1985).
  43. D. J. Amit, H. Gutfreund, and H. Sompolinsky, Statistical mechanics of neural networks near saturation, Ann. Phys. 173, 30 (1987).
  44. D. O. Hebb, The Organization of Behavior: A Neuropsychological Theory (Wiley, New York, NY, 1949).
  45. W. G. Noid, Perspective: Coarse-grained models for biomolecular systems, J. Chem. Phys. 139, 9 (2013).
  46. R. Potestio, C. Peter, and K. Kremer, Computer simulations of soft matter: Linking the scales, Entropy 16, 4199 (2014).
  47. S. Kmiecik, D. Gront, M. Kolinski, L. Wieteska, A. E. Dawid, and A. Kolinski, Coarse-grained protein models and their applications, Chem. Rev. 116, 7898 (2016).
  48. R. J. Cubero, M. Marsili, and Y. Roudi, Multiscale relevance and informative encoding in neuronal spike trains, J. Comput. Neurosci. 48, 85 (2020).
  49. M. Giulini, R. Menichetti, and R. Potestio, A journey through mapping space: Characterising the statistical and metric properties of reduced representations of macromolecules, Eur. Phys. J. B 94, 204 (2021).
  50. S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, Optimization by simulated annealing, Science 220, 671 (1983).
  51. V. Černý, Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm, J. Opt. Theory Appl. 45, 41 (1985).
  52. M. Marsili and Y. Roudi, Quantifying relevance in learning and inference, Phys. Rep. 963, 1 (2022).
  53. A. Haimovici and M. Marsili, Criticality of mostly informative samples: A Bayesian model selection approach, J. Stat. Mech. (2015) P10013.
  54. M. Marsili, I. Mastromatteo, and Y. Roudi, On sampling and modeling complex systems, J. Stat. Mech. (2013) P09003.
  55. M. Mele, R. Covino, and R. Potestio, Information-theoretical measures identify accurate low-resolution representations of protein configurational space, Soft Matter 18, 7064 (2022).
  56. https://doi.org/10.5281/zenodo.13940980

Outline

Information

Phys. Rev. E 111, 044315– Published 21 April, 2025
  • Received 23 October 2024
  • Accepted 24 March 2025
Crossmark - Check for updates button

©2025 American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation