- Physical Review Journals
All JournalsPhysics Magazine
Access Provided by Xinjiang University

Physical Review Fluids

Export citation

Export citation

Choose format for download:

Download Citation
  • Access by Xinjiang University

Probabilistic neural networks for fluid flow surrogate modeling and data recovery

Romit Maulik*

Kai Fukami†

Nesar Ramachandra‡

Koji Fukagata§

Kunihiko Taira∥

  • Argonne Leadership Computing Facility, Argonne National Laboratory, Lemont, Illinois 60439, USA

  • School of Science for Open and Environmental Systems, Keio University, Yokohama 223-8522, Japan

  • High Energy Physics Division, Argonne National Laboratory, Lemont, Illinois 60439, USA

  • Department of Mechanical Engineering, Keio University, Yokohama 223-8522, Japan

  • Department of Mechanical and Aerospace Engineering, University of California, Los Angeles, Los Angeles, California 90095, USA

  • *rmaulik@anl.gov
  • †kai.fukami@keio.jp
  • ‡nramachandra@anl.gov
  • §fukagata@mech.keio.ac.jp
  • ∥ktaira@seas.ucla.edu
PDF

Phys. Rev. Fluids 5, 104401 – Published 8 October, 2020

DOI: https://doi.org/10.1103/PhysRevFluids.5.104401

Abstract

We consider the use of probabilistic neural networks for fluid flow surrogate modeling and data recovery. This framework is constructed by assuming that the target variables are sampled from a Gaussian distribution conditioned on the inputs. Consequently, the overall formulation sets up a procedure to predict the hyperparameters of this distribution which are then used to compute an objective function given training data. We demonstrate that this framework has the ability to provide for prediction confidence intervals based on the assumption of a probabilistic posterior, given an appropriate model architecture and adequate training data. The applicability of the present framework to cases with noisy measurements and limited observations is also assessed. To demonstrate the capabilities of this framework, we consider canonical regression problems of fluid dynamics from the viewpoint of reduced-order modeling and spatial data recovery for four canonical data sets. The examples considered in this study arise from (i) the shallow-water equations, (ii) a two-dimensional cylinder flow, (iii) the wake of a NACA0012 airfoil with a Gurney flap, and (iv) the NOAA sea surface temperature data set. The present results indicate that the probabilistic neural network not only produces a machine-learning-based fluid flow surrogate model but also systematically quantifies the uncertainty therein to assist with model interpretability.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (106)

  1. S. L. Brunton, B. R. Noack, and P. Koumoutsakos, Machine learning for fluid mechanics, Annu. Rev. Fluid Mech. 52, 477 (2020).
  2. M. P. Brenner, J. D. Eldredge, and J. B. Freund, Perspective on machine learning for advancing fluid mechanics, Phys. Rev. Fluids 4, 100501 (2019).
  3. J. N. Kutz, Deep learning in fluid dynamics, J. Fluid Mech. 814, 1 (2017).
  4. K. Fukami, K. Fukagata, and K. Taira, Assessment of supervised machine learning for fluid flows, Theor. Comput. Fluid Dyn. 34, 497 (2020).
  5. S. L. Brunton, M. S. Hemanti, and K. Taira, Special issue on machine learning and data-driven methods in fluid dynamics, Theor. Comput. Fluid Dyn. 34, 333 (2020).
  6. R. Maulik, O. San, J. D. Jacob, and C. Crick, Sub-grid scale model classification and blending through deep learning, J. Fluid Mech. 870, 784 (2019).
  7. M. Gamahara and Y. Hattori, Searching for turbulence models by artificial neural network, Phys. Rev. Fluids 2, 054604 (2017).
  8. R. Maulik and O. San, A neural network approach for the blind deconvolution of turbulent flows, J. Fluid Mech. 831, 151 (2017).
  9. A. Vollant, G. Balarac, and C. Corre, Subgrid-scale scalar flux modeling based on optimal estimation theory and machine-learning procedures, J. Turbul. 18, 854 (2017).
  10. J. Ling and J. Templeton, Evaluation of machine learning algorithms for prediction of regions of high Reynolds averaged Navier Stokes uncertainty, Phys. Fluids 27, 085103 (2015).
  11. E. J. Parish and K. Duraisamy, A paradigm for data-driven predictive modeling using field inversion and machine learning, J. Comput. Phys. 305, 758 (2016).
  12. J. Ling, A. Kurzawski, and J. Templeton, Reynolds averaged turbulence modeling using deep neural networks with embedded invariance, J. Fluid Mech. 807, 155 (2016).
  13. J.-L. Wu, H. Xiao, and E. Paterson, Physics-informed machine learning approach for augmenting turbulence models: A comprehensive framework, Phys. Rev. Fluids 3, 074602 (2018).
  14. R. Matai and P. A. Durbin, Zonal eddy viscosity models based on machine learning, Flow Turbul. Combust. 103, 93 (2019).
  15. L. Zhu, W. Zhang, J. Kou, and Y. Liu, Machine learning methods for turbulence modeling in subsonic flows around airfoils, Phys. Fluids 31, 015105 (2019).
  16. K. Duraisamy, G. Iaccarino, and H. Xiao, Turbulence modeling in the age of data, Annu. Rev. Fluid Mech. 51, 357 (2019).
  17. N. Geneva and N. Zabaras, Quantifying model form uncertainty in Reynolds-averaged turbulence models with Bayesian deep neural networks, J. Comput. Phys. 383, 125 (2019).
  18. N. Geneva and N. Zabaras, Multi-fidelity generative deep learning turbulent flows, arXiv:2006.04731.
  19. K. Taira, M. S. Hemati, S. L. Brunton, Y. Sun, K. Duraisamy, S. Bagheri, S. Dawson, and C. Yeh, Modal analysis of fluid flows: Applications and outlook, AIAA J. 58, 998 (2020).
  20. K. Taira, M. S. Hemati, and L. S. Ukeiley, Modal analysis of fluid flows: Introduction to the virtual collection, AIAA J. 58, 991 (2020).
  21. P. R. Vlachas, W. Byeon, Z. Y. Wan, T. P. Sapsis, and P. Koumoutsakos, Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks, Proc. R. Soc. A 474, 20170844 (2018).
  22. S. E. Ahmed, O. San, A. Rasheed, and T. Iliescu, A long short-term memory embedding for hybrid uplifted reduced order models, Physica D: Non. Phen. 409, 132471 (2020).
  23. R. Maulik, A. Mohan, B. Lusch, S. Madireddy, and P. Balaprakash, Time-series learning of latent-space dynamics for reduced-order model closure, Physica D 405, 132368 (2020).
  24. A. T. Mohan, D. Daniel, M. Chertkov, and D. Livescu, Compressed convolutional LSTM: An efficient deep learning framework to model high fidelity 3D turbulence, arXiv:1903.00033.
  25. A. T. Mohan and D. V. Gaitonde, A deep learning based approach to reduced order modeling for turbulent flow control using LSTM neural networks, arXiv:1804.09269.
  26. Q. Wang, N. Ripamonti, and J. S. Hesthaven, Recurrent neural network closure of parametric POD-Galerkin reduced-order models based on the Mori-Zwanzig formalism, J. Comput. Phys. 410, 109402 (2020).
  27. B. Lusch, J. N. Kutz, and S. L. Brunton, Deep learning for universal linear embeddings of nonlinear dynamics, Nat. Commun. 9, 4950 (2018).
  28. F. J. Gonzalez and M. Balajewicz, Learning low-dimensional feature dynamics using deep convolutional recurrent autoencoders, arXiv:1808.01346.
  29. J. Xu and K. Duraisamy, Multi-level convolutional autoencoder networks for parametric prediction of spatio-temporal dynamics, Comput. Methods Appl. Mech. Eng. 372, 113379 (2020).
  30. R. Maulik, B. Lusch, and P. Balaprakash, Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders, arXiv:2002.00470.
  31. M. Cheng, F. Fang, C. C. Pain, and I. M. Navon, An advanced hybrid deep adversarial autoencoder for parameterized nonlinear fluid flow modelling, arXiv:2003.10547.
  32. K. Hasegawa, K. Fukami, T. Murata, and K. Fukagata, Machine-learning-based reduced-order modeling for unsteady flows around bluff bodies of various shapes, Theor. Comput. Fluid Dyn. 34, 367 (2020).
  33. C. W. Rowley and S. T. M. Dawson, Model reduction for flow analysis and control, Annu. Rev. Fluid Mech. 49, 387 (2017).
  34. J. Borggaard, T. Iliescu, and Z. Wang, Artificial viscosity proper orthogonal decomposition, Math. Comput. Model. 53, 269 (2011).
  35. X. Xie, M. Mohebujjaman, L. G. Rebholz, and T. Iliescu, Data-driven filtered reduced order modeling of fluid flows, SIAM J. Sci. Comput. 40, B834 (2018).
  36. E. Rezaian and M. Wei, A hybrid stabilization approach for reduced-order models of compressible flows with shock-vortex interaction, Int. J. Numer. Methods Eng. 121, 1629 (2020).
  37. O. San and J. Borggaard, Principal interval decomposition framework for POD reduced-order modeling of convective Boussinesq flows, Int. J. Numer. Methods Fluids 78, 37 (2015).
  38. O. San and R. Maulik, Extreme learning machine for reduced order modeling of turbulent geophysical flows, Phys. Rev. E 97, 042322 (2018).
  39. S. Sheriffdeen, J. C. Ragusa, J. E. Morel, M. L. Adams, and T. Bui-Thanh, Accelerating PDE-constrained inverse solutions with deep learning and reduced order models, arXiv:1912.08864.
  40. J. N. Kutz, S. L. Brunton, B. W. Brunton, and J. L. Proctor, Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems (SIAM, Philadelphia, 2016).
  41. J. N. Kutz, X. Fu, and S. L. Brunton, Multiresolution dynamic mode decomposition, SIAM J. Appl. Dyn. Syst. 15, 713 (2016).
  42. O. San and R. Maulik, Neural network closures for nonlinear model order reduction, Adv. Comput. Math. 44, 1717 (2018).
  43. K. Fukami, Y. Nabae, K. Kawai, and K. Fukagata, Synthetic turbulent inflow generator using machine learning, Phys. Rev. Fluids 4, 064603 (2019).
  44. N. B. Erichson, L. Mathelin, Z. Yao, S. L. Brunton, M. W. Mahoney, and J. N. Kutz, Shallow neural networks for fluid flow reconstruction with limited sensors, Proc. R. Soc. A 476, 20200097 (2020).
  45. K. Fukami, K. Fukagata, and K. Taira, Super-resolution reconstruction of turbulent flows with machine learning, J. Fluid Mech. 870, 106 (2019).
  46. K. Fukami, K. Fukagata, and K. Taira, Proceedings of the 11th International Symposium on Turbulence and Shear Flow Phenomena (TSFP11), Southampton, 2019, available at http://www.tsfp-conference.org/proceedings/2019/208.pdf (unpublished).
  47. R. Onishi, D. Sugiyama, and K. Matsuda, Super-resolution simulation for real-time prediction of urban micrometeorology, Sci. Online Lett. Atmos. 15, 178 (2019).
  48. B. Liu, J. Tang, H. Huang, and X.-Y. Lu, Deep learning methods for super-resolution reconstruction of turbulent flows, Phys. Fluids 32, 025105 (2020).
  49. K. Fukami, K. Fukagata, and K. Taira, Machine learning based spatio-temporal super resolution reconstruction of turbulent flows, arXiv:2004.11566.
  50. M. Milano and P. Koumoutsakos, Neural network modeling for near wall turbulent flow, J. Comput. Phys. 182, 1 (2002).
  51. T. Murata, K. Fukami, and K. Fukagata, Nonlinear mode decomposition with convolutional neural networks for fluid dynamics, J. Fluid Mech. 882, A13 (2020).
  52. K. Fukami, T. Nakamura, and K. Fukagata, Convolutional neural network based hierarchical autoencoder for nonlinear mode decomposition of fluid field data, Phys. Fluids 32, 095110 (2020).
  53. T. Bui-Thanh, M. Damodaran, and K. Willcox, Aerodynamic data reconstruction and inverse design using proper orthogonal decomposition, AIAA J. 42, 1505 (2004).
  54. T. Suzuki and Y. Hasegawa, Estimation of turbulent channel flow at Reτ=100 based on the wall measurement using a simple sequential approach, J. Fluid Mech. 830, 760 (2006).
  55. M. Morimoto, K. Fukami, and K. Fukagata, Experimental velocity data estimation for imperfect particle images using machine learning, arXiv:2005.00756.
  56. S. Cai, S. Zhou, C. Xu, and Q. Gao, Dense motion estimation of particle images via a convolutional neural network, Exp. Fluids 60, 60 (2019).
  57. Z. Deng, C. He, Y. Liu, and K. C. Kim, Super-resolution reconstruction of turbulent velocity fields using a generative adversarial network-based artificial intelligence framework, Phys. Fluids 31, 125111 (2019).
  58. J. Kim and C. Lee, Prediction of turbulent heat transfer using convolutional neural networks, J. Fluid Mech. 882, A18 (2020).
  59. G. E. P. Box and G. C. Tiao, Bayesian Inference in Statistical Analysis (Wiley, New York, 2011), Vol. 40.
  60. R. M. Neal, Bayesian Learning for Neural Networks (Springer Science + Business Media, New York, 2012), Vol. 118.
  61. B. Mohebali, A. Tahmassebi, A. Meyer-Baese, and A. H. Gandomi, in Handbook of Probabilistic Models, edited by P. Samui, D. Tien Bui, S. Chakraborty, and R. C. Deo (Butterworth-Heinemann, Oxford, 2020), pp. 347–367.
  62. Y. Gal, Uncertainty in deep learning, Ph.D. thesis, University of Cambridge, 2016.
  63. E. Goan and C. Fookes, Bayesian neural networks: An introduction and survey, Case Stud. App. Bay. Data Sci. 2259, 45 (2020).
  64. G. E. Hinton and D. van Camp, in Proceedings of the Sixth Annual Conference on Computational Learning Theory, Santa Cruz, 1993, edited by L. Pitt (Association for Computing Machinery, New York, 1993), pp. 5–13.
  65. C. Blundell, J. Cornebise, K. Kavukcuoglu, and D. Wierstra, in Proceedings of the 32nd International Conference on Machine Learning, edited by F. Bach and D. Blei (JMLR, Brookline, 2015), Vol. 37, pp. 1613–1622.
  66. A. Graves, in Proceedings of the 24th International Conference on Advances in Neural Information Processing Systems, edited by J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger (Curran, Red Hook, 2011), pp. 2348–2356.
  67. D. P. Kingma and M. Welling, Auto-encoding variational Bayes, arXiv:1312.6114.
  68. Y. Gal and Z. Ghahramani, Dropout as a Bayesian approximation, arXiv:1506.02157.
  69. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting, J. Mach. Learn. Res. 15, 1929 (2014).
  70. C.-K. Lu, S. C.-H. Yang, X. Hao, and P. Shafto, Interpretable deep Gaussian processes with moments, arXiv:1905.10963.
  71. A. Damianou and N. Lawrence, Proceedings of the 16th International Conference on Artificial Intelligence and Statistics, Scottsdale, 2013 (JMLR, Brookline, 2013), Vol. 31, pp. 207–215.
  72. R. Alexander and D. Giannakis, Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques, Physica D 409, 132520 (2020).
  73. S. Santini and A. D. Bimbo, Recurrent neural networks can be trained to be maximum a posteriori probability classifiers, Neural Netw. 8, 25 (1995).
  74. C. M. Bishop, Mixture density networks, Aston University, Neural Computing Research Group Report No. NCRG/94/004, 1994 (unpublished).
  75. D. Ormoneit and V. Tresp, in Proceedings of the Ninth International Conference on Advances in Neural Information Processing Systems, Denver, 1996, edited by M. C. Mozer, M. I. Jordan, and T. Petsche (MIT Press, Cambridge, 1997), pp. 542–548.
  76. F. Yang, J. Soriano, T. Kubo, and K. Ikeda, in Proceedings of the 24th International Conference on Neural Information Processing, Guangzhou, 2017, edited by D. Liu, S. Xie, Y. Li, D. Zhao, and E.-S. M. El-Alfy, Lecture Notes in Computer Science Vol. 10637 (Springer, Cham, 2017), pp. 878–885.
  77. O. Makansi, E. Ilg, O. Cicek, and T. Brox, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, Piscataway, 2019), pp. 7144–7153.
  78. C. P. Martin and J. Torresen, RoboJam: A musical mixture density network for collaborative touchscreen interaction, Lect. Notes Comput. Sci. 10783, 161 (2018).
  79. A. D'Isanto and K. L. Polsterer, Photometric redshift estimation via deep learning. Generalized and pre-classification-less, image based, fully probabilistic redshifts, Astron. Astrophys. 609, A111 (2018).
  80. M. Lucey, Y.-S. Ting, N. S. Ramachandra, and K. Hawkins, From the inner to outer Milky Way: A photometric sample of 2.6 million red clump stars, Mon. Not. R. Astron. Soc. 495, 3087 (2020).
  81. G. J. McLachlan and K. E. Basford, Mixture models: Inference and applications to clustering J. R. Stat. Soc. C 38, 384 (1989).
  82. D. J. C. Mackay, Probable networks and plausible predictions — A review of practical Bayesian methods for supervised neural networks, Netw.: Comput. Neural Syst. 6, 469 (1995).
  83. F. Wenzel, K. Roth, B. S. Veeling, J. Świątkowski, L. Tran, S. Mandt, J. Snoek, T. Salimans, R. Jenatton, and S. Nowozin, How good is the Bayes posterior in deep neural networks really? arXiv:2002.02405.
  84. M. A. Nielsen, Neural Networks and Deep Learning (Determination, San Francisco, 2015), Vol. 2018.
  85. D. P. Kingma and J. Ba, Proceedings of the Third International Conference for Learning Representations, San Diego, 2015 (unpublished).
  86. J. Bergstra, D. Yamins, and D. Cox, in Proceedings of the 30th International Conference on Machine Learning, Atlanta, 2013, edited by S. Dasgupta and D. McAllester (JMLR, Brookline, 2013), Vol. 28.
  87. E. Brochu, V. Cora, and N. de Freitas, A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning, University of British Columbia Report No. TR-2009-023, 2009 (unpublished).
  88. R. Maulik, K. Fukami, N. Ramachandra, K. Fukagata, and K. Taira, https://github.com/Romit-Maulik/Probabilistic_ML_Fluids.
  89. Available at https://www.esrl.noaa.gov/psd/.
  90. K. Taira, S. L. Brunton, S. T. M. Dawson, C. W. Rowley, T. Colonius, B. J. McKeon, O. T. Schmidt, S. Gordeyev, V. Theofilis, and L. S. Ukeiley, Modal analysis of fluid flows: An overview, AIAA J. 55, 4013 (2017).
  91. D. D. Kosambi, Statistics in function space, J. Indian Math. Soc. 7, 76 (1943).
  92. G. Berkooz, P. Holmes, and J. L. Lumley, The proper orthogonal decomposition in the analysis of turbulent flows, Annu. Rev. Fluid Mech. 25, 539 (1993).
  93. X.-D. Liu, S. Osher, and T. Chan, Weighted essentially non-oscillatory schemes, J. Comput. Phys. 115, 200 (1994).
  94. E. Hairer, S. P. Nørsett, and G. Wanner, Solving Ordinary Differential Equations I: Nonstiff Problems, Springer Series in Computational Mathematics Vol. 8 (Springer, Berlin, 1993).
  95. R. Maulik and O. San, Resolution and energy dissipation characteristics of implicit LES and explicit filtering models for compressible turbulence, Fluids 2, 14 (2017).
  96. A. Mendible, S. L. Brunton, A. Y. Aravkin, W. Lowrie, and J. N. Kutz, Dimensionality reduction and reduced order modeling for traveling wave physics, Theor. Comput. Fluid Dyn. 34, 385 (2020).
  97. J. L. Callaham, K. Maeda, and S. L. Brunton, Robust flow reconstruction from limited measurements via sparse representation, Phys. Rev. Fluids 4, 103907 (2019).
  98. R. Maulik, R. Egele, B. Lusch, and P. Balaprakash, Recurrent neural network architecture search for geophysical emulation, arXiv:2004.10928.
  99. R. Everson and L. Sirovich, Karhunen-Loeve procedure for gappy data, J. Opt. Soc. Am. A 12, 1657 (1995).
  100. K. Taira and T. Colonius, The immersed boundary method: A projection approach, J. Comput. Phys. 225, 2118 (2007).
  101. T. Colonius and K. Taira, A fast immersed boundary method using a nullspace approach and multi-domain far-field boundary conditions, Comput. Methods Appl. Mech. Eng. 197, 2131 (2008).
  102. M. G. Meena, K. Taira, and K. Asai, Airfoil wake modification with gurney flap at low Reynolds number, AIAA J. 56, 1348 (2018).
  103. M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics informed deep learning (Part II): Data-driven discovery of nonlinear partial differential equations, arXiv:1711.10566.
  104. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE 86, 2278 (1998).
  105. Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y. Philip, A comprehensive survey on graph neural networks, IEEE Trans. Neural Netw. Learn. Syst. (2020), doi:10.1109/TNNLS.2020.2978386.
  106. N. Trask, R. G. Patel, B. J. Gross, and P. J. Atzberger, GMLS–Nets: A framework for learning from unstructured data, arXiv:1909.05371.

Outline

Information

Phys. Rev. Fluids 5, 104401– Published 8 October, 2020
  • Received 8 May 2020
  • Accepted 16 September 2020
Crossmark - Check for updates button

©2020 American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation