Framework for End-to-End Tuning and Regression for a High Performance MPI Library on Modern Supercomputers
Loading...
Date
2020-05
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
The Ohio State University
Abstract
The Message Passing Interface (MPI) is a popular parallel programming model for developing parallel scientific applications. Collective operations defined in the MPI standard offer a convenient abstraction to implement group communication operations. Owing to their ease of use and performance portability, collective operations are used across various scientific domains. The MVAPICH2 library implements the collective operations using various algorithms. Each algorithm performs differently based on various factors such as message size, system size, CPU type, interconnect type, topology, etc. Thus, a question arises from the implementer's perspective--how can MVAPICH2 be optimized so that it utilizes the best algorithm for any combination of the factors mentioned above? To address this problem, we have created a "tuning" framework---a set of programs that compare the performance of various algorithms by varying the above factors and selects the best one for a given combination. Experimental results with the Tuning Framework show performance improvements of up to 79%. In addition to the Tuning Framework, this work introduces a Regression Framework which simplifies the understanding of change in performance between versions of MVAPICH2.
Description
Keywords
High Performance Computing, MPI, Collectives, Tuning, MVAPICH2