On the improvement of a scalable sparse direct solver for unsymmetrical linear equations

M. Serdar Celebi*, Ahmet Duran, Figen Oztoprak, Mehmet Tuncel, Bora Akaydin

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

This paper focuses on the application level improvements in a sparse direct solver specifically used for large-scale unsymmetrical linear equations resulting from unstructured mesh discretization of coupled elliptic/hyperbolic PDEs. Existing sparse direct solvers are designed for distributed server systems taking advantage of both distributed memory and processing units. We conducted extensive numerical experiments with three state-of-the-art direct linear solvers that can work on distributed-memory parallel architectures; namely, MUMPS (MUMPS solver website, http://graal.ens-lyon.fr/MUMPS), WSMP (Technical Report TR RC-21886, IBM, Watson Research Center, Yorktown Heights, 2000), and SUPERLU_DIST (ACM Trans Math Softw 29(2):110–140, 2003). The performance of these solvers was analyzed in detail, using advanced analysis tools such as Tuning and Analysis Utilities (TAU) and Performance Application Programming Interface (PAPI). The performance is evaluated with respect to robustness, speed, scalability, and efficiency in CPU and memory usage. We have determined application level issues that we believe they can improve the performance of a distributed-shared memory hybrid variant of this solver, which is proposed as an alternative solver [SuperLU_MCDT (Many-Core Distributed)] in this paper. The new solver utilizing the MPI/OpenMP hybrid programming is specifically tuned to handle large unsymmetrical systems arising in reservoir simulations so that higher performance and better scalability can be achieved for a large distributed computing system with many nodes of multicore processors. Two main tasks are accomplished during this study: (i) comparisons of public domain solver algorithms; existing state-of-the-art direct sparse linear system solvers are investigated and their performance and weaknesses based on test cases are analyzed, (ii) improvement of direct sparse solver algorithm (SuperLU_MCDT) for many-core distributed systems is achieved. We provided results of numerical tests that were run on up to 16,384 cores, and used many sets of test matrices for reservoir simulations with unstructured meshes. The numerical results showed that SuperLU_MCDT can outperform SuperLU_DIST 3.3 in terms of both speed and robustness.

Original languageEnglish
Pages (from-to)1852-1904
Number of pages53
JournalJournal of Supercomputing
Volume73
Issue number5
DOIs
Publication statusPublished - 1 May 2017

Bibliographical note

Publisher Copyright:
© 2016, Springer Science+Business Media New York.

Keywords

  • Large scale simulations
  • Linear equations
  • Many-core distributed solver
  • Numerical factorization
  • Parallel linear direct solver
  • Reservoir simulations
  • Scalability
  • Sparse direct solver
  • SuperLU
  • Symbolic factorization

Fingerprint

Dive into the research topics of 'On the improvement of a scalable sparse direct solver for unsymmetrical linear equations'. Together they form a unique fingerprint.

Cite this