- Physical Review Journals
All JournalsPhysics Magazine
Access Provided by Xinjiang University

Physical Review E

Export citation

Export citation

Choose format for download:

Download Citation
  • Access by Xinjiang University

Temporal stimulus segmentation by reinforcement learning in populations of spiking neurons

Luisa Le Donne*

Lik Chun Chan

Robert Urbanczik† and Walter Senn

Giancarlo La Camera‡

  • Department of Neurobiology and Behavior and Graduate Programs in Neuroscience, Stony Brook University, Stony Brook, New York 11794, USA

  • Department of Neurobiology and Behavior, Center for Neural Circuit Dynamics, and Graduate Programs in Neuroscience, Stony Brook University, Stony Brook, New York 11794, USA

  • Department of Neurobiology and Behavior, Center for Neural Circuit Dynamics, AI Innovation Institute, Center for Advanced Computational Science, and Graduate Programs in Neuroscience, Stony Brook University, Stony Brook, New York 11794, USA

  • *Present address: Space Biomedical Centre, University of Rome Tor Vergata, Rome, Italy.
  • †Deceased.
  • ‡Contact author: giancarlo.lacamera@stonybrook.edu
PDF

Phys. Rev. E 113, 024406 – Published 25 February, 2026

DOI: https://doi.org/10.1103/4mwt-2yb2

Abstract

Learning to detect, identify, or select stimuli is an essential requirement of many behavioral tasks. In real-life situations, relevant and nonrelevant stimuli are often embedded in a continuous sensory stream, presumably represented by different segments of neural activity. Here we introduce a spiking network model that can discover action-relevant stimuli in an unsegmented sensory stream of spike trains. The model uses a biologically plausible plasticity rule and learns from the reinforcement of correct decisions taken at the right time. Learning is fully online and is faster for larger population size; it allows for a wide spectrum of neural-encoding strategies and can segment cortical spike patterns recorded from behaving animals. Based on these results, the proposed model provides a biologically plausible framework for reinforcement learning in the absence of prior information on the identity, relevance, and timing of input stimuli embedded in a continuous spatiotemporal stream.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (81)

  1. P. Dayan and N. D. Daw, Cognit. Affective, Behav. Neurosci. 8, 429 (2008).
  2. X.-J. Wang, Neuron 60, 215 (2008).
  3. J. Zhang, R. Bogacz, and P. Holmes, J. Math. Psychol. 53, 231 (2009).
  4. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed., Adaptive Computation and Machine Learning Series (MIT Press, Cambridge, MA, 2018).
  5. D. Zeithamova, M. L. Mack, K. Braunlich, T. Davis, C. A. Seger, M. T. R. van Kesteren, and A. Wutz, J. Neurosci. 39, 8259 (2019).
  6. J.-P. Pfister, T. Toyoizumi, D. Barber, and W. Gerstner, Neural Comput. 18, 1318 (2006).
  7. R. Urbanczik and W. Senn, Nat. Neurosci. 12, 250 (2009).
  8. J. N. Reynolds and J. R. Wickens, Neural Netw. 15, 507 (2002).
  9. M. Oram, M. Wiener, R. Lestienne, and B. Richmond, J. Neurophysiol. 81, 3021 (1999).
  10. M. C. Wiener and B. J. Richmond, Biosystems 67, 295 (2002).
  11. J. H. Macke, P. Berens, A. S. Ecker, A. S. Tolias, and M. Bethge, Neural Comput. 21, 397 (2009).
  12. C. Samuelsen, M. Gardner, and A. Fontanini, Neuron 74, 410 (2012).
  13. D. B. Katz, S. A. Simon, and M. A. Nicolelis, J. Neurosci. 21, 4478 (2001).
  14. A. Jezzini, L. Mazzucato, G. La Camera, and A. Fontanini, J. Neurosci. 33, 18966 (2013).
  15. N. K. Horst and M. Laubach, Front. Neurosci. 7, 56 (2013).
  16. L. Mazzucato, A. Fontanini, and G. La Camera, J. Neurosci. 35, 8214 (2015).
  17. M. Shadlen and W. Newsome, J. Neurosci. 18, 3870 (1998).
  18. M. M. Churchland, B. M. Yu, J. P. Cunningham, L. P. Sugrue, M. R. Cohen, G. S. Corrado, W. T. Newsome, A. M. Clark, P. Hosseini, B. B. Scott, D. C. Bradley, M. A. Smith, A. Kohn, J. A. Movshon, K. M. Armstrong, T. Moore, S. W. Chang, L. H. Snyder, S. G. Lisberger, N. J. Priebe, et al., Nat Neurosci 13, 369 (2010).
  19. P. Dayan and L. F. Abbott, Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems (Massachusetts Institute of Technology Press, Cambridge, MA, 2001).
  20. E. Zohary, M. N. Shadlen, and W. T. Newsome, Nature (London) 370, 140 (1994).
  21. E. Schneidman, M. J. Berry, II, R. Segev, and W. Bialek, Nature (London) 440, 1007 (2006).
  22. F. Montani, R. A. A. Ince, R. Senatore, E. Arabzadeh, M. E. Diamond, and S. Panzeri, Philos. Trans. A Math. Phys. Eng. Sci. 367, 3297 (2009).
  23. I. E. Ohiorhenuan, F. Mechler, K. P. Purpura, A. M. Schmid, Q. Hu, and J. D. Victor, Nature (London) 466, 617 (2010).
  24. S. Yu, H. Yang, H. Nakahara, G. S. Santos, D. Nikolić, and D. Plenz, J. Neurosci. 31, 17514 (2011).
  25. E. Ganmor, R. Segev, and E. Schneidman, Proc. Natl. Acad. Sci. USA 108, 9679 (2011).
  26. B. Doiron, A. Litwin-Kumar, R. Rosenbaum, G. K. Ocker, and K. Josić, Nat. Neurosci. 19, 383 (2016).
  27. L. Mazzucato, A. Fontanini, and G. La Camera, Front. Syst. Neurosci. 10, 11 (2016).
  28. G. Corrado and K. Doya, J. Neurosci. 27, 8178 (2007).
  29. J. I. Gold and M. N. Shadlen, Annu. Rev. Neurosci. 30, 535 (2007).
  30. D. Lee and X.-J. Wang, in Neuroeconomics: Decision Making and the Brain, edited by P. Glimcher, C. Camerer, E. Fehr, and R. Poldrack (Academic Press, London, UK, 2009), Chap. 31, pp. 481–502.
  31. D. Green and J. Swets, Signal Detection Theory and Psychophysics (Wiley, New York, 1966).
  32. K. H. Britten, M. N. Shadlen, W. T. Newsome, and J. A. Movshon, J. Neurosci. 12, 4745 (1992).
  33. A. K. Churchland, R. Kiani, and M. N. Shadlen, Nat. Neurosci. 11, 693 (2008).
  34. I. M. Park, M. L. R. Meister, A. C. Huk, and J. W. Pillow, Nat. Neurosci. 17, 1395 (2014).
  35. A. N. Hampton, P. Bossaerts, and J. P. O'Doherty, J. Neurosci. 26, 8360 (2006).
  36. D. Zeithamova, W. T. Maddox, and D. M. Schnyer, J. Neurosci. 28, 13194 (2008).
  37. W. Schultz, J. Neurophysiol. 80, 1 (1998).
  38. A. J. Yu and P. Dayan, Neural Netw. 15, 719 (2002).
  39. H. S. Seung, Neuron 40, 1063 (2003).
  40. E. M. Izhikevich, Cereb. Cortex 17, 2443 (2007).
  41. N. Frémaux, H. Sprekeler, and W. Gerstner, J. Neurosci. 30, 13326 (2010).
  42. X. Xie and H. S. Seung, Phys. Rev. E 69, 041909 (2004).
  43. J. Werfel, X. Xie, and H. S. Seung, Neural Comput. 17, 2699 (2005).
  44. I. R. Fiete, M. S. Fee, and H. S. Seung, J. Neurophysiol. 98, 2038 (2007).
  45. J. Friedrich, R. Urbanczik, and W. Senn, Neural Comput. 22, 1698 (2010).
  46. B. B. Averbeck, P. E. Latham, and A. Pouget, Nat. Rev. Neurosci. 7, 358 (2006).
  47. M. Shadlen and W. Newsome, Curr. Opin. Neurobiol. 4, 569 (1994).
  48. G. Holt, W. Softky, C. Koch, and R. Douglas, J. Neurophysiol. 75, 1806 (1996).
  49. J. Friedrich, R. Urbanczik, and W. Senn, Int. J. Neur. Syst. 24, 1450002 (2014).
  50. R. Bogacz, E. Brown, J. Moehlis, P. Holmes, and J. D. Cohen, Psychol Rev. 113, 700 (2006).
  51. T. Hartley, C. Lever, N. Burgess, and J. J. O'Keefe, Phil. Trans. R. Soc. B 369, 20120510 (2014).
  52. J. O'Keefe and M. L. Recce, Hippocampus 3, 317 (1993).
  53. W. E. Skaggs, B. L. McNaughton, M. A. Wilson, and C. A. Barnes, Hippocampus 6, 149 (1996).
  54. A. Ponce-Alvarez, V. Nácher, R. Luna, A. Riehle, and R. Romo, J. Neurosci. 32, 11956 (2012).
  55. L. M. Jones, A. Fontanini, B. F. Sadacca, P. Miller, and D. B. Katz, Proc. Natl. Acad. Sci. USA 104, 18772 (2007).
  56. H. Shimazaki, K. Sadeghi, T. Ishikawa, Y. Ikegaya, and T. Toyoizumi, Sci. Rep. 5, 9821 (2015).
  57. T. Fawcett, Pattern Recogn. Lett. 27, 861 (2006).
  58. T. Saito and M. Rehmsmeier, PLoS One 10, e0118432 (2015).
  59. L. Rabiner, Proc. IEEE 77, 257 (1989).
  60. M. Abeles, H. Bergman, I. Gat, I. Meilijson, E. Seidemann, N. Tishby, and E. Vaadia, Proc. Natl. Acad. Sci. USA 92, 8616 (1995).
  61. K. Maboudi, E. Ackermann, L. W. de Jong, B. E. Pfeiffer, D. Foster, K. Diba, and C. Kemere, eLife 7, e34467 (2018).
  62. L. Mazzucato, G. La Camera, and A. Fontanini, Nat. Neurosci. 22, 787 (2019).
  63. S. J. Gershman and D. M. Blei, J. Math. Psychol. 56, 1 (2012).
  64. Z. Chen, Comput. Intell. Neurosci. 2013, 251905 (2013).
  65. G. Celeux and J. Diebolt, Comput. Stat. Q. 2, 73 (1985).
  66. R. M. Neal and G. E. Hinton, in Learning in Graphical Models, edited by M. I. Jordan (Kluwer Academic Press, Amsterdam, 1998), pp. 355–368.
  67. G. Mongillo and S. Deneve, Neural Comput. 20, 1706 (2008).
  68. D. Kappel, B. Nessler, and W. Maass, PLoS Comput. Biol. 10, e1003511 (2014).
  69. N. Frémaux, H. Sprekeler, and W. Gerstner, PLoS Comput. Biol. 9, e1003024 (2013).
  70. R. Legenstein, D. Pecevski, and W. Maass, PLoS Comput. Biol. 4, e1000180 (2008).
  71. S. Klampfl and W. Maass, J. Neurosci. 33, 11515 (2013).
  72. T. Masquelier, R. Guyonneau, and S. J. Thorpe, Neural Comput. 21, 1259 (2009).
  73. T. Masquelier, R. Guyonneau, and S. J. Thorpe, PLoS ONE 3, e1377 (2008).
  74. R. Gütig, Science 351, aab4113 (2016).
  75. J. Friedrich, R. Urbanczik, and W. Senn, PLoS Comput. Biol. 7, e1002092 (2011).
  76. M. Rigotti, D. Ben Dayan Rubin, S. E. Morrison, C. D. Salzman, and S. Fusi, NeuroImage 52, 833 (2010).
  77. A. Genovesio, P. J. Brasted, A. R. Mitz, and S. P. Wise, Neuron 47, 307 (2005).
  78. J. Tanji, K. Shima, and H. Mushiake, Trends Cognit. Sci. 11, 528 (2007).
  79. M. Rigotti, D. B. D. Rubin, X.-J. Wang, and S. Fusi, Front. Comput. Neurosci. 4, 24 (2010).
  80. G. La Camera, S. Bouret, and B. J. Richmond, Front. Neurosci. 12, 165 (2018).
  81. F. A. Mansouri, D. J. Freedman, and M. J. Buckley, Nat. Rev. Neurosci. 21, 595 (2020).

Outline

Information

Phys. Rev. E 113, 024406– Published 25 February, 2026
  • Received 15 July 2021
  • Accepted 23 January 2026
Article part of CHORUS
Crossmark - Check for updates button

©2026 American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation