- Access by Xinjiang University
On-line learning of unrealizable tasks
Phys. Rev. E 60, 5902 – Published 1 November, 1999
DOI: https://doi.org/10.1103/PhysRevE.60.5902
Abstract
The dynamics of on-line learning is investigated for structurally unrealizable tasks in the context of two-layer neural networks with an arbitrary number of hidden neurons. Within a statistical mechanics framework, a closed set of differential equations describing the learning dynamics can be derived, for the general case of unrealizable isotropic tasks. In the asymptotic regime one can solve the dynamics analytically in the limit of a large number of hidden neurons, providing an analytical expression for the residual generalization error, the optimal and critical asymptotic training parameters, and the corresponding prefactor of the generalization error decay.
References (11)
- G. Cybenko, Math. Control Signals and Syst. 2, 303 (1988).
- H. S. Seung, H. Sompolinsky, and N. Tishby, Phys. Rev. A 45, 6056 (1993).
- T. L. H. Watkin, A. Rau, and M. Biehl, Rev. Mod. Phys. 65, 499 (1993).
- On-Line Learning in Neural Networks, edited by D. Saad (Newton Institute, Cambridge University Press, Cambridge, 1998).
- M. Biehl and H. Schwarze, J. Phys. A 28, 643 (1995).
- D. Saad and S. A. Solla, Phys. Rev. Lett. 74, 4337 (1995); Phys. Rev. E 52, 4225 (1995).
- A. West and D. Saad, Phys. Rev. E 56, 3426 (1997).
- M. Biehl, P. Riegler, and C. Wohel, J. Phys. A 29, 4767 (1996).
- T. K. Leen, B. Schottky, and D. Saad, Advances in Neural Information Systems, edited by M. I. Jordan, M. J. Kearns, and S. A. Solla (MIT Press, Cambridge, MA, 1998), Vol. 10, p. 301; T. K. Leen, B. Schottky, and D. Saad, Phys. Rev. E 59, 985 (1999).
- T. Kato, A Short Introduction to Perturbation Theory for Linear Operators (Springer-Verlag, Berlin, 1982).
- C. Cohen-Tannoudji, Quantum Mechanics (Wiley, New York, 1977).