- Physical Review Journals
All JournalsPhysics Magazine
Access Provided by Xinjiang University

Physical Review Fluids

Export citation

Export citation

Choose format for download:

Download Citation
  • Access by Xinjiang University

Embedding hard physical constraints in neural network coarse-graining of three-dimensional turbulence

Arvind T. Mohan*

Nicholas Lubbers

Misha Chertkov

Daniel Livescu

  • Center for Nonlinear Studies Computational Physics and Methods Group, Los Alamos National Laboratory, Los Alamos 87545, United States

  • Information Sciences Group, Los Alamos National Laboratory, Los Alamos 87545, United States

  • Program in Applied Mathematics, University of Arizona, Tucson 85721, United States

  • Computational Physics and Methods Group, Los Alamos National Laboratory, Los Alamos 87545, United States

  • *Corresponding author: arvindm@lanl.gov
PDF

Phys. Rev. Fluids 8, 014604 – Published 31 January, 2023

DOI: https://doi.org/10.1103/PhysRevFluids.8.014604

Abstract

In recent years, deep learning approaches have shown much promise in modeling complex systems in the physical sciences. A major challenge in deep learning of partial differential equations is enforcing physical constraints and boundary conditions. In this work, we propose a general framework to directly embed the notion of an incompressible fluid into convolutional neural networks, and apply this to coarse-graining of turbulent flow. These physics-embedded neural networks leverage interpretable strategies from numerical methods and computational fluid dynamics to enforce physical laws and boundary conditions by taking advantage the mathematical properties of the underlying equations. We demonstrate results on three-dimensional fully developed turbulence, showing that this technique drastically improves local conservation of mass, without sacrificing performance according to several other metrics characterizing the fluid flow.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (57)

  1. A. T. Mohan, D. Tretiak, M. Chertkov, and D. Livescu Spatio-temporal deep learning models of 3D turbulence with physics informed diagnostics. J. Turbul. 21, 484 (2020).
  2. D. Xiao, F. Fang, C. Pain, and G. Hu, Non-intrusive reduced-order modelling of the Navier-Stokes equations based on RBF Interpolation, Int. J. Numer. Methods Fluids 79, 580 (2015).
  3. J. S. Hesthaven and S. Ubbiali, Non-intrusive reduced order modeling of nonlinear problems using neural networks, J. Comput. Phys. 363, 55 (2018).
  4. D. Xiao, C. Heaney, F. Fang, L. Mottet, R. Hu, D. Bistrian, E. Aristodemou, I. Navon, and C. Pain, A domain decomposition non-intrusive reduced order model for turbulent flows, Comput. Fluids 182, 15 (2019).
  5. J. Yu, C. Yan, and M. Guo, Non-intrusive reduced-order modeling for fluid problems: A brief review, Proc. Inst. Mech. Eng. Part G: J. Aerosp. Enging. 233, 5896 (2019).
  6. K. Fukami, K. Fukagata, and K. Taira, Super-resolution reconstruction of turbulent flows with machine learning, J. Fluid Mech. 870, 106 (2019).
  7. K. Fukami, Y. Nabae, K. Kawai, and K. Fukagata, Synthetic turbulent inflow generator using machine learning, Phys. Rev. Fluids 4, 064603 (2019).
  8. J. Kim and C. Lee, Prediction of turbulent heat transfer using convolutional neural networks, J. Fluid Mech. 882, A18 (2020).
  9. M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys. 378, 686 (2019).
  10. J.-L. Wu, K. Kashinath, A. Albert, D. Chirila, Prabhat et al., Enforcing statistical constraints in generative adversarial networks for modeling chaotic dynamical systems, J. Comput. Phys. 406, 109209 (2020).
  11. N. B. Erichson, M. Muehlebach, and M. W. Mahoney, Physics-informed autoencoders for lyapunov-stable fluid flow prediction, arXiv:1905.10866 (2019).
  12. R. Wang, K. Kashinath, M. Mustafa, A. Albert, and R. Yu, Towards physics-informed deep learning for turbulent flow prediction, arXiv:1911.08655 (2019).
  13. B. Lusch, J. N. Kutz, and S. L. Brunton, Deep learning for universal linear embeddings of nonlinear dynamics, Nat. Commun. 9, 4950 (2018).
  14. Y. Bengio, Practical recommendations for gradient-based training of deep architectures, in Neural Networks: Tricks of the Trade (Springer, Berlin, 2012), pp. 437–478.
  15. L. N. Smith, Cyclical learning rates for training neural networks, in Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) (IEEE, New York, 2017), pp. 464–472.
  16. T. M. Breuel, The effects of hyperparameters on SGD training of neural networks, arXiv:1508.02788 (2015).
  17. D. F. Gordon and M. Desjardins, Evaluation and selection of biases in machine learning, Springer Mach. Learn. 20, 5 (1995).
  18. T. M. Mitchell, The Need for Biases in Learning Generalizations, Rutgers Computer Science Tech report (Rutgers, New Jersey, 1980).
  19. N. Cohen and A. Shashua, Inductive bias of deep convolutional networks through pooling geometry, arXiv:1605.06743 (2016).
  20. A. Gaier and D. Ha, Weight agnostic neural networks, arXiv:1906.04358 (2019).
  21. H. Tennekes, J. L. Lumley, J. L. Lumley et al., A First Course in Turbulence (MIT Press, Cambridge, MA, 1972).
  22. L. Lu, X. Meng, Z. Mao, and G. E. Karniadakis, DeepXDE: A deep learning library for solving differential equations, arXiv:1907.04502 (2019).
  23. R. King, O. Hennigh, A. Mohan, and M. Chertkov, From deep to physics-informed learning of turbulence: Diagnostics, arXiv:1810.07785 (2018).
  24. A. T. Mohan and D. V. Gaitonde, A deep learning based approach to reduced order modeling for turbulent flow control using LSTM neural networks, arXiv:1804.09269 (2018).
  25. S. Wiewel, M. Becher, and N. Thuerey, Latent space physics: Towards learning the temporal evolution of fluid flow, in Computer Graphics Forum (Wiley Online Library, Hoboken, NJ, 2019), Vol. 38, pp. 71–82.
  26. B. Kim, V. C. Azevedo, N. Thuerey, T. Kim, M. Gross, and B. Solenthaler, Deep fluids: A generative network for parameterized fluid simulations, in Computer Graphics Forum (Wiley Online Library, Hoboken, NJ, 2019), Vol. 38, pp. 59–70.
  27. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning (MIT Press, Cambridge, MA, 2016).
  28. Y. LeCun, Y. Bengio, and G. Hinton, Deep learning, Nature (London) 521, 436 (2015).
  29. A. Krizhevsky, I. Sutskever, and G. E. Hinton, Imagenet classification with deep convolutional neural networks, in Advances in Neural Information Processing Systems (Lake Tahoe, 2012), pp. 1097–1105.
  30. Z. Long, Y. Lu, X. Ma, and B. Dong, PDE-net: Learning PDEs from data, arXiv:1710.09668 (2017).
  31. Z. Long, Y. Lu, and B. Dong, Pde-net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network, J. Comput. Phys. 399, 108925 (2019).
  32. B. Dong, Q. Jiang, and Z. Shen, Image restoration: Wavelet frame shrinkage, nonlinear evolution PDEs, and beyond, Multiscale Model. Simul. 15, 606 (2017).
  33. J.-F. Cai, B. Dong, S. Osher, and Z. Shen, Image restoration: Total variation, wavelet frames, and beyond, J. Am. Math. Soc. 25, 1033 (2012).
  34. E. Oja, Neural networks, principal components, and subspaces, Int. J. Neur. Syst. 01, 61 (1989).
  35. F. J. Gonzalez and M. Balajewicz, Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems, arXiv:1808.01346 (2018).
  36. T. Murata, K. Fukami, and K. Fukagata, Nonlinear mode decomposition with convolutional neural networks for fluid dynamics, J. Fluid Mech. 882, A13 (2020).
  37. O. Hennigh, Lat-net: Compressing lattice Boltzmann flow simulations using deep neural networks, arXiv:1705.09036.
  38. W. Chen, A. R. Tan, and A. L. Ferguson, Collective variable discovery and enhanced sampling using autoencoders: Innovations in network architecture and error function design, J. Chem. Phys. 149, 072312 (2018).
  39. W. Wang and R. Gómez-Bombarelli, Coarse-graining auto-encoders for molecular dynamics, npj Computat. Mater. 5, 125 (2019).
  40. J. Ling, A. Kurzawski, and J. Templeton, Reynolds averaged turbulence modelling using deep neural networks with embedded invariance, J. Fluid Mech. 807, 155 (2016).
  41. J. H. Ferziger, Numerical Methods for Engineering Qpplication (Wiley, New York, 1981), Vol. 1.
  42. D. B. Spalding, A novel finite difference formulation for differential expressions involving both first and second derivatives, Int. J. Numer. Methods Eng. 4, 551 (1972).
  43. E. Fadlun, R. Verzicco, P. Orlandi, and J. Mohd-Yusof, Combined immersed-boundary finite-difference methods for three-dimensional complex flow simulations, J. Comput. Phys. 161, 35 (2000).
  44. Y.-H. Tseng and J. H. Ferziger, A ghost-cell immersed boundary method for flow in complex geometry, J. Comput. Phys. 192, 593 (2003).
  45. P. A. Berthelsen and O. M. Faltinsen, A local directional ghost cell approach for incompressible viscous flow problems with irregular boundaries, J. Comput. Phys. 227, 4354 (2008).
  46. G. J. Hirasaki and J. Hellums, A general formulation of the boundary conditions on the vector potential in three-dimensional hydrodynamics, Q. Appl. Math. 26, 331 (1968).
  47. O. Biro and K. Preis, On the use of the magnetic vector potential in the finite-element analysis of three-dimensional eddy currents, IEEE Trans. Magn. 25, 3145 (1989).
  48. D. Daniel, D. Livescu, and J. Ryu, Reaction analogy based forcing for incompressible scalar turbulence, Phys. Rev. Fluids 3, 094602 (2018).
  49. S. Pope, Turbulent Flows (Cambridge University Press, Cambridge, 2000).
  50. A. Perry and M. Chong, A description of eddying motions and flow patterns using critical-point concepts, Annu. Rev. Fluid Mech. 19, 125 (1987).
  51. Y. Tian, F. Jaberi, and D. Livescu, Density effects on the post-shock turbulence structure and dynamics, J. Fluid Mech. 880, 935 (2019).
  52. M. Chertkov, A. Pumir, and B. I. Shraiman, Lagrangian tetrad dynamics and the phenomenology of turbulence, Phys. Fluids 11, 2394 (1999).
  53. M. Sakurada and T. Yairi, Anomaly detection using autoencoders with nonlinear dimensionality reduction, in Proceedings of the MLSDA 2014 2nd Workshop on Machine Learning for Sensory Data Analysis (ACM Press, Gold Coast, Australia, 2014), pp. 4–11.
  54. Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, Dynamic graph cnn for learning on point clouds, arXiv:1801.07829 (2018).
  55. M. Petersen and D. Livescu, Forcing for statistically stationary compressible isotropic turbulence, Phys. Fluids 22, 116101 (2010).
  56. Y. Morinishi, T. S. Lund, O. V. Vasilyev, and P. Moin, Fully conservative higher order finite difference schemes for incompressible flow, J. Comput. Phys. 143, 90 (1998).
  57. M. R. Visbal and D. V. Gaitonde, On the use of higher-order finite-difference schemes on curvilinear and deforming meshes, J. Comput. Phys. 181, 155 (2002).

Outline

Information

Phys. Rev. Fluids 8, 014604– Published 31 January, 2023
  • Received 17 February 2020
  • Accepted 23 November 2022
Article part of CHORUS
Crossmark - Check for updates button

©2023 American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation