Evaluating the Maturity of OpenFOAM Simulations on GPGPU for Bio-fluid Applications

We investigated the challenges facing CFD solvers as applied to bio-medical fluid flow simulations and in particular the OpenFOAM 2.1.1 solver, icoFoam, for the large penta-diagonal matrices coming from the simulation of blood flow in arteries with a structured mesh domain in PRACE-3IP project at TGCC Curie (a modern Tier-0 system) (see [1] and references therein). We generated a structured mesh by using blockMesh as a mesh generator tool. To decompose the generated mesh, we employed the decomposePar tool. After the decomposition, we used icoFoam as a flow simulator/solver tool. We achieved scaled speed-up for large matrices up to 64 million x 64 million matrices and speed-up up to 16384 cores on Curie. The sloshing of blood in heart is important to understand heart rate turbulence, potential damage of the sloshing to walls of vessels and heart attack. Therefore, in this study, more realistic bio-fluid mechanics applications will be tested towards petascaling/exascaling these OpenFOAM applications. We will test OpenFOAM “icoFoam” simulator with various iterative solvers such as diagonal incomplete LU preconditioned bi-conjugate gradient, generalised geometric algebraic multi-grid and incomplete cholesky preconditioned conjugate gradient, in addition to direct solvers such as distributed SuperLU (see [2]). Also, we will test several flow simulators such as nonNewtonianIcoFoam and pisoFoam. nonNewtonianIcoFoam is used to simulate non-newtonian flows while pisoFoam is for incompressible turbulent flow. We will use snappyHexMesh tool to construct unstructured mesh in arbitrary shapes such as real artery geometries, as needed. We will compare the performance, scalability and robustness of OpenFOAM on GPGPU for future exascale computing potential and limitations. [1] A. Duran, M.S. Celebi, S. Piskin, and M. Tuncel, Scalability of OpenFOAM For Bio-medical Flow Simulations, Partnership for Advanced Computing in Europe, PRACE White Paper 162, 30 March 2014 [2] X. S. Li, J. W. Demmel, J. R. Gilbert, L. Grigori, M. Shao, and I. Yamazaki, SuperLU Users’ Guide, Tech. Report UCB, Computer Science Division, University of California, Berkeley, CA, 1999, update: 2011