George Bosilca
Innovative Computing Laboratory
Electrical Engineering and Computer Science Department,
University of Tennessee, Knoxville, TN, USA
bosilca@eecs.utk.edu
Today, multi/many core systems have become prevalent, with architectures more or less exotic and heterogeneous. The overall theoretical computational power of the new generation processors has thus greatly increased, but their programmability still lacks certainty. The many changes in the newest architectures have come so rapidly that we are still deficient in taking advantage of all the new features, in terms of high performance libraries and applications. Simultaneously, application requirements grow at least at the same pace. Obviously, more computations require more data in order to feed the deepest processor pipelines. More data means either faster access to the memory or faster access
to the network. But the improvement in access speed to all types of memory (network included) lags behind the increase in computational power. As a result, while extracting the right performance of the current and next generation architectures is still a challenge, it is compulsory to increase the efficiency of the current parallel programming paradigms.
Simultaneously, increasing the size of parallel machines triggers an increase in fault tolerance requirements. While the fault management and recovery topic has been thoughtfully studied over the last decade, recent changes in the number and distribution of the processor's cores have raised some interesting questions. Which fault tolerant approach fits best to the peta-scale environments is still debated, but few of these approaches show interesting performances at scale or a low degree of intrusion in the application code. Eventually, the right answer might be somewhere in between, a dynamic combination of several of these methods, strictly based on the application's properties and the hardware environment.
As expected, all these changes guarantee a highly dynamic (and exciting from a research point of view), high performance arena over the next few years. New mathematical algorithms will have to emerge in order to take advantage of these unbalanced architectures, new programming approaches will have to be established to help these algorithms, and the next generations of computer scientists will have to be fluent in understanding these architectures and competent in understanding the best programming paradigm that fits them.
How MPI will adapt to fit into this con icting environment is still an open question. Over the last few years, MPI has been a very successful parallel programming paradigm, partially due to its apparent simplicity to express basic message exchange patterns and partially to the fact that it increases the productivity of the programmers and the parallel machines. Whatever the future of MPI will be, these two features should remain an indispensable part of its new direction of development.
