- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Physics-inspired transformer quantum states via latent imaginary-time evolution

Kimihiro Yamazaki1,2,*, Itsushi Sakata3, Takuya Konishi1,3, and Yoshinobu Kawahara1,3

  • 1Graduate School of Information Science and Technology, The University of Osaka, 1-5 Yamadaoka, Suita, Osaka, Japan
  • 2Fujitsu Research, Fujitsu Limited, 4-1-1 Kamikodanaka, Nakahara-ku, Kawasaki, Kanagawa, Japan
  • 3Center for Advanced Intelligence Project, RIKEN, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, Japan

  • *Contact author: k-yamazaki@ist.osaka-u.ac.jp
PDF

Phys. Rev. Research 8, 033032 – Published 9 July, 2026

DOI: https://doi.org/10.1103/bjxb-8tsk

Abstract

Neural quantum states (NQS) are powerful ansätze in the variational Monte Carlo framework, yet their architectures are often treated as black boxes. We propose a physically transparent framework in which NQS are treated as neural approximations to latent imaginary-time evolution. This viewpoint suggests that standard transformer-based NQS (TQS) architectures correspond to physically unmotivated effective Hamiltonians dependent on imaginary time in a latent space. Building on this interpretation, we introduce physics-inspired transformer quantum states, which enforce a static effective Hamiltonian by sharing weights across layers and improve propagation accuracy via Trotter-Suzuki decompositions without increasing the number of variational parameters. For the frustrated J1−J2 Heisenberg model, our ansätze achieve accuracies comparable to or exceeding state-of-the-art TQS while using substantially fewer variational parameters. This study demonstrates that reinterpreting the deep network structure as a latent cooling process enables a more physically grounded, systematic, and compact design, thereby bridging the gap between black-box expressivity and physically transparent construction.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (37)

  1. M. Imada, A. Fujimori, and Y. Tokura, Metal-insulator transitions, Rev. Mod. Phys. 70, 1039 (1998).
  2. G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science 355, 602 (2017).
  3. W. L. McMillan, Ground state of liquid He4, Phys. Rev. 138, A442 (1965).
  4. L. L. Viteritti, R. Rende, and F. Becca, Transformer variational wave functions for frustrated quantum spin systems, Phys. Rev. Lett. 130, 236401 (2023).
  5. L. L. Viteritti, R. Rende, A. Parola, S. Goldt, and F. Becca, Transformer wave function for two dimensional frustrated magnets: Emergence of a spin-liquid phase in the Shastry-Sutherland model, Phys. Rev. B 111, 134411 (2025).
  6. M. Geier, K. Nazaryan, T. Zaklama, and L. Fu, Self-attention neural network for solving correlated electron problems in solids, Phys. Rev. B 112, 045119 (2025).
  7. Y. Gu, W. Li, H. Lin, B. Zhan, R. Li, Y. Huang, D. He, Y. Wu, T. Xiang, M. Qin, L. Wang, and D. Lv, Solving the Hubbard model with neural quantum states, Nat. Commun. (2026).
  8. R. Rende, L. L. Viteritti, L. Bardone, F. Becca, and S. Goldt, A simple linear algebra identity to optimize large-scale neural network quantum states, Commun. Phys. 7, 260 (2024).
  9. M. Gell-Mann and F. Low, Bound states in quantum field theory, Phys. Rev. 84, 350 (1951).
  10. R. Blankenbecler, D. J. Scalapino, and R. L. Sugar, Monte Carlo calculations of coupled boson-fermion systems. I, Phys. Rev. D 24, 2278 (1981).
  11. G. Sugiyama and S. Koonin, Auxiliary field Monte-Carlo for quantum many-body ground states, Ann. Phys. 168, 1 (1986).
  12. G. Carleo, Y. Nomura, and M. Imada, Constructing exact representations of quantum many-body systems with deep neural networks, Nat. Commun. 9, 5322 (2018).
  13. S. Lie, Theorie der Transformationsgruppen (B. G. Teubner, Leipzig, 1888), Vol. 1.
  14. H. F. Trotter, On the product of semi-groups of operators, Proc. Am. Math. Soc. 10, 545 (1959).
  15. G. Strang, On the construction and comparison of difference schemes, SIAM J. Numer. Anal. 5, 506 (1968).
  16. M. Suzuki, Fractal decomposition of exponential operators with applications to many-body theories and Monte Carlo simulations, Phys. Lett. A 146, 319 (1990).
  17. S. Blanes and P. Moan, Practical symplectic partitioned Runge–Kutta and Runge–Kutta–Nyström methods, J. Comput. Appl. Math. 142, 313 (2002).
  18. A. Chen and M. Heyl, Empowering deep neural quantum states through efficient optimization, Nat. Phys. 20, 1476 (2024).
  19. S. Blanes, F. Casas, and A. Murua, Splitting methods for differential equations, Acta Numer. 33, 1 (2024).
  20. M. Dehghani, S. Gouws, O. Vinyals, J. Uszkoreit, and L. Kaiser, Universal transformers, in Proceedings of the International Conference on Learning Representations (ICLR, 2019).
  21. J. L. Elman, Finding structure in time, Cognit. Sci. 14, 179 (1990).
  22. Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, ALBERT: A lite BERT for self-supervised learning of language representations, in Proceedings of the International Conference on Learning Representations (ICLR, 2020).
  23. K. Yamazaki, I. Sakata, T. Konishi, and Y. Kawahara, Source Code for “Physics-inspired transformer quantum states via latent imaginary-time evolution”, Zenodo, 2026, https://doi.org/10.5281/zenodo.18334851.
  24. R. Rende and L. L. Viteritti, Are queries and keys always relevant? A case study on transformer wave functions, Mach. Learn.: Sci. Technol. 6, 010501 (2025).
  25. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need, in Proceedings of the 31st International Conference on Neural Information Processing Systems (Curran Associates, Inc., Red Hook, NY, 2017), pp. 6000–6010.
  26. G. Carleo, K. Choo, D. Hofmann, J. E. Smith, T. Westerhout, F. Alet, E. J. Davis, S. Efthymiou, I. Glasser, S.-H. Lin, M. Mauri, G. Mazzola, C. B. Mendl, E. van Nieuwenburg, O. O’Reilly, H. Théveniaut, G. Torlai, F. Vicentini, and A. Wietek, NetKet: A machine learning toolkit for many-body quantum systems, SoftwareX 10, 100311 (2019).
  27. F. Vicentini, D. Hofmann, A. Szabó, D. Wu, C. Roth, C. Giuliani, G. Pescia, J. Nys, V. Vargas-Calderón, N. Astrakhantsev, and G. Carleo, NetKet 3: Machine learning toolbox for many-body quantum systems, SciPost Phys. Codebases 7 (2022).
  28. J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang, JAX: Composable transformations of Python+NumPy programs, 2018, https://github.com/jax-ml/jax.
  29. J. Heek, A. Levskaya, A. Oliver, M. Ritter, B. Rondepierre, A. Steiner, and M. van Zee, Flax: A neural network library and ecosystem for JAX, 2024, https://github.com/google/flax.
  30. D. Häfner and F. Vicentini, Mpi4jax: Zero-copy MPI communication of JAX arrays, J. Open Source Software 6, 3419 (2021).
  31. https://netket.readthedocs.io/en/latest/tutorials/ViT-wave-function.html.
  32. R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. Duvenaud, Neural ordinary differential equations, in Proceedings of the 32nd International Conference on Neural Information Processing Systems (Curran Associates, Inc., Red Hook, NY, 2018), pp. 6572–6583.
  33. Y. Lu, Z. Li, D. He, Z. Sun, B. Dong, T. Qin, L. Wang, and T.-Y. Liu, Understanding and improving transformer from a multi-particle dynamic system point of view, in Proceedings of the ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations (ICLR, 2020).
  34. K. Heun, Neue Methode zur approximativen Integration der Differentialgleichungen einer unabhängigen Veränderlichen, Z. Math. Phys. 45, 23 (1900).
  35. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al., PaLM: Scaling language modeling with Pathways, J. Mach. Learn. Res. 24, 240 (2023).
  36. M. Dehghani, J. Djolonga, B. Mustafa, P. Padlewski, J. Heek, J. Gilmer, A. P. Steiner, M. Caron, R. Geirhos, I. Alabdulmohsin, et al., Scaling vision transformers to 22 billion parameters, in Proceedings of the 40th International Conference on Machine Learning (PMLR, 2023), Vol. 202, pp. 7480–7512.
  37. J. S. Anderson, M. Nakata, R. Igarashi, K. Fujisawa, and M. Yamashita, The second-order reduced density matrix method and the two-dimensional Hubbard model, Comput. Theor. Chem. 1003, 22 (2013).

Outline

Information

Phys. Rev. Research 8, 033032– Published 9 July, 2026
  • Received 4 February 2026
  • Accepted 10 June 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation