- Access by Xinjiang University
Attention-based neural network emulators for multiprobe data vectors. III. Modeling the next generation surveys
Phys. Rev. D 113, 103536 – Published 26 May, 2026
DOI: https://doi.org/10.1103/ks4r-25km
Abstract
Machine learning can accelerate cosmological inferences that involve many sequential evaluations of computationally expensive data vectors. Previous works in this series have examined how machine learning architectures impact emulator accuracy and training time for optical shear and galaxy clustering two-point function. In this final manuscript, we explore neural network performance when emulating cosmic microwave background (CMB) temperature and polarization power spectra. We maximize the volume of applicability in the parameter space of our emulators within the standard -cold-dark-matter model while ensuring that errors are below cosmic variance. Relative to standard multi-layer perceptron architectures, we find that the dot-product-attention mechanism reduces the number of outliers among testing cosmologies, defined as the fraction of testing points with relative to camb outputs, for a wide range of training set sizes. Such precision enables attention-based emulators to be directly applied to real data without requiring any additional correction via importance sampling. Combined with pre-processing techniques and optimized activation and loss functions, attention-based models can meet the precision criteria set by current and future CMB and lensing experiments. For each of Planck, Simons Observatory, CMB-S4, and CMB-HD, we find the fraction of outlier points to be less than 10% with around to training data vectors. We further explore the applications of these methods to supernova distance, weak lensing, and galaxy clustering, as well as alternative architectures and pre-processing techniques.
Physics Subject Headings (PhySH)
Article Text
References (56)
- P. Ade et al. (Simons Observatory Collaboration), J. Cosmol. Astropart. Phys. 02 (2019) 056.
- M. Abitbol et al. (Simons Observatory Collaboration), J. Cosmol. Astropart. Phys. 08 (2025) 034.
- K. Abazajian et al., arXiv:2203.08024.
- N. Sehgal et al., Bull. Am. Astron. Soc. 51, 1 (2019).
- A. Lewis, A. Challinor, and A. Lasenby, Astrophys. J. 538, 473 (2000).
- J. Lesgourgues, arXiv:1104.2932.
- D. Blas, J. Lesgourgues, and T. Tram, J. Cosmol. Astropart. Phys. 07 (2011) 034.
- J. a. Rebouças, D. H. F. de Souza, K. Zhong, V. Miranda, and R. Rosenfeld, J. Cosmol. Astropart. Phys. 02 (2025) 024.
- A. Hajian, Phys. Rev. D 75, 083525 (2007).
- M. Bonici, F. Bianchini, and J. Ruiz-Zapatero, Open J. Astrophys. 7, 10 (2023).
- F. Ge et al. (SPT-3G Collaboration), Phys. Rev. D 111, 083534 (2025).
- D. Piras and A. Spurio Mancini, Open J. Astrophys. 6, 20 (2023).
- J. U. Lange, Mon. Not. R. Astron. Soc. 525, 3181 (2023).
- F. Feroz, M. P. Hobson, and M. Bridges, Mon. Not. R. Astron. Soc. 398, 1601 (2009).
- M. Kamionkowski, Phys. Rev. D 104, 063512 (2021).
- A. Spurio Mancini, D. Piras, J. Alsing, B. Joachimi, and M. P. Hobson, Mon. Not. R. Astron. Soc. 511, 1771 (2022).
- H. T. Jense, I. Harrison, E. Calabrese, A. Spurio Mancini, B. Bolliet, J. Dunkley, and J. C. Hill, RAS Tech. Instrum. 4, rzaf002 (2025).
- G.-J. Wang, C. Cheng, Y.-Z. Ma, J.-Q. Xia, A. Abebe, and A. Beesham, Astrophys. J. Suppl. Ser. 268, 7 (2023).
- B. Bolliet, A. Spurio Mancini, J. C. Hill, M. Madhavacheril, H. T. Jense, E. Calabrese, and J. Dunkley, Mon. Not. R. Astron. Soc. 531, 1351 (2024).
- K. Zhong, E. Saraivanov, J. Caputi, V. Miranda, S. S. Boruah, T. Eifler, and E. Krause, Phys. Rev. D 111, 123519 (2025).
- E. Saraivanov, K. Zhong, V. Miranda, S. S. Boruah, T. Eifler, and E. Krause, Phys. Rev. D 111, 123520 (2025).
- E. Krause et al., arXiv:2105.13548.
- Y. Bengio, P. Simard, and P. Frasconi, IEEE Trans. Neural Networks 5, 157 (1994).
- R. Pascanu, T. Mikolov, and Y. Bengio, arXiv:1211.5063.
- K. He, X. Zhang, S. Ren, and J. Sun, arXiv:1512.03385.
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, arXiv:1706.03762.
- A. Katharopoulos, A. Vyas, N. Pappas, and F. Fleuret, in Proceedings of the International Conference on Machine Learning (ICML) (PMLR, 2020).
- A. Vyas, A. Katharopoulos, and F. Fleuret, Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS) (Curran Associates, Inc., 2020).
- R. Li, J. Su, C. Duan, and S. Zheng, arXiv:2007.14902.
- R. Dolga, L. Maystre, M. Cobzarenco, and D. Barber, arXiv:2402.17512.
- N. Kitaev, Ł. Kaiser, and A. Levskaya, arXiv:2001.04451.
- S. Zhai, W. Talbott, N. Srivastava, C. Huang, H. Goh, R. Zhang, and J. Susskind, arXiv:2105.14103.
- J. Alsing, H. Peiris, J. Leja, C. Hahn, R. Tojeiro, D. Mortlock, B. Leistedt, B. D. Johnson, and C. Conroy, Astrophys. J. Suppl. Ser. 249, 5 (2020).
- A. Nygaard, E. B. Holm, S. Hannestad, and T. Tram, J. Cosmol. Astropart. Phys. 05 (2023) 025.
- J. R. Bond, A. H. Jaffe, and L. E. Knox, Astrophys. J. 533, 19 (2000).
- H. K. Eriksen, J. B. Jewell, C. Dickinson, A. J. Banday, K. M. Górski, and C. R. Lawrence, Astrophys. J. 676, 10 (2008).
- N. Aghanim et al. (Planck Collaboration), Astron. Astrophys. 641, A5 (2020).
- H. Prince and J. Dunkley, Phys. Rev. D 105, 023518 (2022).
- C. Devon Lin and B. Tang, arXiv:2203.06334.
- N. Aghanim et al. (Planck Collaboration), Astron. Astrophys. 641, A6 (2020).
- W. Hu and M. J. White, Astrophys. J. 479, 568 (1997).
- J. Adamo, H.-J. Huang, and T. Eifler, Phys. Rev. D 110, 123517 (2024).
- J. Su, Y. Lu, S. Pan, A. Murtadha, B. Wen, and Y. Liu, arXiv:2104.09864.
- E. Calabrese et al. (ACT Collaboration), J. Cosmol. Astropart. Phys. 11 (2025) 063.
- https://github.com/CosmoLike.
- A. MacInnis, N. Sehgal, and M. Rothermel, Phys. Rev. D 109, 063527 (2024).
- J. Lesgourgues, arXiv:1104.2934.
- N. Aghanim et al. (Planck Collaboration), Astron. Astrophys. 641, A1 (2020).
- M. J. Mortonson, W. Hu, and D. Huterer, Phys. Rev. D 79, 023004 (2009).
- S. Kiranyaz, O. Avci, O. Abdeljaber, T. Ince, M. Gabbouj, and D. J. Inman, arXiv:1905.03554.
- S. Pandya, Y. Yang, N. Van Alfen, J. Blazek, and R. Walters, arXiv:2504.05235.
- J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (The MIT Press, Cambridge, MA, 1992).
- J. B. Orjuela-Quintana, D. Sapone, and S. Nesseris, Phys. Rev. D 113, 063526 (2026).
- D. J. Bartlett, B. D. Wandelt, M. Zennaro, P. G. Ferreira, and H. Desmond, Astron. Astrophys. 686, A150 (2024).
- J. B. Orjuela-Quintana, S. Nesseris, and W. Cardona, Phys. Rev. D 107, 083520 (2023).
- M. Cranmer, arXiv:2305.01582.