Papers by Victor Eijkhout
Chapman and Hall/CRC eBooks, Nov 26, 2013
Tables 77.1 to Table .5 present a list of freely available software for the solution of linear al... more Tables 77.1 to Table .5 present a list of freely available software for the solution of linear algebra problems. The interest is in software for high-performance computers that is available in "open source" form on the Web for solving problems in numerical linear algebra, specifically dense, sparse direct and iterative systems, and sparse iterative eigenvalue problems.
Journal of Parallel and Distributed Computing, Aug 1, 2018
Helpful discussions with Kazushige Goto and John McCalpin are gratefully acknowledged. Thanks to ... more Helpful discussions with Kazushige Goto and John McCalpin are gratefully acknowledged. Thanks to Dan Stanzione for his notes on cloud computing, Ernie Chan for his notes on scheduling of block algorithms, and John McCalpin for his analysis of the top500. Thanks to Elie de Brauwer, Susan Lindsey, and Lorenzo Pesce for proofreading and many comments. Edmond Chow wrote the chapter on Molecular Dynamics. Robert van de Geijn contributed several sections on dense linear algebra. Contents
Teaching MPI from Mental Models
The Message Passing Interface (MPI) is the de facto standard for programming large scale parallel... more The Message Passing Interface (MPI) is the de facto standard for programming large scale parallelism, with up to millions of individual processes. Its dominant paradigm of Single Program Multiple Data (SPMD) programming is different from threaded and multicore parallelism, to an extent that students have a hard time switching models. In contrast to threaded programming, which allows for a view of the execution with central control and a central repository of data, SPMD programming has a symmetric model where all processes are active all the time, and none is priviliged in any sense, and where data is distributed.This model is counterintuitive to the novice parallel programmer, so care needs to be taken how to instill the proper ‘mental model'.We identify problems with the currently common way of teaching MPI, and propose a way that is geared to explicit reinforcing the symmetric model. Additionally, we teach starting from realistic scenarios, rather than writing artificial code just to exercise a newly-learned routine.This motivation implies that we reverse the commonly used order of presenting MPI routines, starting with collectives, and later introducing point-to-point routines only as support for certain symmetric operations, avoiding the process-to-process model.
5. Remaining Topics
Society for Industrial and Applied Mathematics eBooks, 1994
Blas
Discrete mathematics and its applications, Nov 26, 2013
LAPACK Working Note 76: Algorithmic Bombardment for the Iterative Solution of Linear Systems: A Poly-Iterative Approach
Preconditioners for the Conjugate Gradient Method 1 1.1 where Λ'^1 = Л^'-Л^'Л', 1 ,' ' Л^'. In th... more Preconditioners for the Conjugate Gradient Method 1 1.1 where Λ'^1 = Л^'-Л^'Л', 1 ,' ' Л^'. In the next step Л^' can then be used to eliminate the second row. The matrix blocks A { £¿ are called thepz'uois от pivot blocks. Gaussian elimination, by blocks or by scalars, can be formulated as a de composition /< ••• <:\
We present an experimental investigation of the performance of MPI derived datatypes. For message... more We present an experimental investigation of the performance of MPI derived datatypes. For messages up to the megabyte range most schemes perform comparably to each other and to manual copying into a regular send buffer. However, for large messages the internal buffering of MPI causes differences in efficiency. The optimal scheme is a combination of packing and derived types.
4. Related Issues
Society for Industrial and Applied Mathematics eBooks, 1994
Tools and Guidelines for Job Bundling on Modern Supercomputers
Practice and Experience in Advanced Research Computing, 2021
As many scientific computation tasks focus on solving large-scale and computationally intensive p... more As many scientific computation tasks focus on solving large-scale and computationally intensive problems, a wide range of problems involving High-Throughput Computing (HTC) paradigms and data-oriented algorithms emerge. To solve these problems, we review and recommend some simple workflows for users to bundle their serial or parallel jobs in this paper, including the native way with the Linux scheduler, the numactl method for processes or shared memory management, and the job array feature of job scheduler. We also introduce several convenient job bundling tools that TACC develops, such as ibrun, Launcher, Pylauncher, and Launcher-GPU. Some basic practice guidelines are added for users to choose appropriate workflows and tools when job bundling is required.
LAPACK Working Note 77: Basic Concepts for Distributed Sparse Linear Algebra Operations
This manual describes the use of PETSc 2.0 for the numerical solution of partial differential equ... more This manual describes the use of PETSc 2.0 for the numerical solution of partial differential equations and related problems on high-performance computers. The Portable, Extensible Toolkit for Scientific computation (PETSc) is a suite of data structures and routines that provide the building blocks for the implementation of large-scale application codes on parallel (and serial) computers. PETSc 2.0 uses the MPI standard for all message-passing communication. PETSc includes an expanding suite of parallel linear and -nonlinear equation solvers that may be used in application codes ,written in Fortran, C, and C++. PETSc provides many of the mechanisms needed within parallel application codes, such as simple parallel matrix and vector assembly routines that allow the overlap of communication and computation. In addition, PETSc includes growing support for distributed arrays. The library is organized hierarchically, enabling users to employ the level of abstraction that is most appropriate for a particular problem. By using techniques of object-oriented programming, PETSc provides enormous flexibility for users. PETSc is a sophisticated set of software tools; as such, for some users it initially has a much steeper learning curve than a simple subroutine library. In particular, for individuals without some computer science background or experience programming in C, Pascal, or C++, it may require a large amount of time to take full advantage of the features that enable efficient software use. However, the power of the PETSc design and the algorithms it incorporates make the efficient implementation of many application codes much simpler than "rolling them" yourself. For many simple (or even relatively complicated) tasks a package such as Matlab is often the best tool; PETSc is not intended for the classes of problems for which effective Matlab code can be written. Since PETSc is still under development, small changes in usage and calling sequences of PETSc routines will continue to occur. Although keeping one's code up to date can be somewhat annoying, all PETSc users will be rewarded in the long run with a cleaner, better designed,
This manual describes the use of PETSc 2.0 for the numerical solution of partial differential equ... more This manual describes the use of PETSc 2.0 for the numerical solution of partial differential equations and related problems on high-performance computers. The Portable, Extensible Toolkit for Scientific computation (PETSc) is a suite of data structures and routines that provide the building blocks for the implementation of large-scale application codes on parallel (and serial) computers. PETSc 2.0 uses the MPI standard for all message-passing communication. PETSc includes an expanding suite of parallel linear and -nonlinear equation solvers that may be used in application codes ,written in Fortran, C, and C++. PETSc provides many of the mechanisms needed within parallel application codes, such as simple parallel matrix and vector assembly routines that allow the overlap of communication and computation. In addition, PETSc includes growing support for distributed arrays. The library is organized hierarchically, enabling users to employ the level of abstraction that is most appropriate for a particular problem. By using techniques of object-oriented programming, PETSc provides enormous flexibility for users. PETSc is a sophisticated set of software tools; as such, for some users it initially has a much steeper learning curve than a simple subroutine library. In particular, for individuals without some computer science background or experience programming in C, Pascal, or C++, it may require a large amount of time to take full advantage of the features that enable efficient software use. However, the power of the PETSc design and the algorithms it incorporates make the efficient implementation of many application codes much simpler than "rolling them" yourself. For many simple (or even relatively complicated) tasks a package such as Matlab is often the best tool; PETSc is not intended for the classes of problems for which effective Matlab code can be written. Since PETSc is still under development, small changes in usage and calling sequences of PETSc routines will continue to occur. Although keeping one's code up to date can be somewhat annoying, all PETSc users will be rewarded in the long run with a cleaner, better designed,
Topics in Parallel and Distributed Computing, 2015
This chapter introduces a number of basic concepts in parallel programming using the Game of Life... more This chapter introduces a number of basic concepts in parallel programming using the Game of Life as an example. No specific programming systems will be taught here, but this chapter should leave the student with a basic understanding of the fundamental concepts. There are some pen-and-paper exercises.
This manual describes the use of PETSc for the numerical solution of partial differential equatio... more This manual describes the use of PETSc for the numerical solution of partial differential equations and related problems on high-performance computers. The Portable, Extensible Toolkit for Scientific Computation (PETSc) is a suite of data structures and routines that provide the building blocks for the implementation of large-scale application codes on parallel (and serial) computers. PETSc uses the MPI standard for all message-passing communication. PETSc includes an expanding suite of parallel linear, nonlinear equation solvers and time integrators that may be used in application codes written in Fortran, C, C++, Python, and MATLAB (sequential). PETSc provides many of the mechanisms needed within parallel application codes, such as parallel matrix and vector assembly routines. The library is organized hierarchically, enabling users to employ the level of abstraction that is most appropriate for a particular problem. By using techniques of object-oriented programming, PETSc provides enormous flexibility for users. PETSc is a sophisticated set of software tools; as such, for some users it initially has a much steeper learning curve than a simple subroutine library. In particular, for individuals without some computer science background, experience programming in C, C++ or Fortran and experience using a debugger such as gdb or dbx, it may require a significant amount of time to take full advantage of the features that enable efficient software use. However, the power of the PETSc design and the algorithms it incorporates may make the efficient implementation of many application codes simpler than "rolling them" yourself. • LINPACK -dense matrix factorization and solve; converted to C using f2c and then hand-optimized for small matrix sizes, for block matrix data structures; • MINPACK -see page 121, sequential matrix coloring routines for finite difference Jacobian evaluations; converted to C using f2c; • SPARSPAK -see page 79, matrix reordering routines, converted to C using f2c; • libtfs -the efficient, parallel direct solver developed by Henry Tufo and Paul Fischer for the direct solution of a coarse grid problem (a linear system with very few degrees of freedom per processor). PETSc interfaces to the following external software: • ADIFOR -automatic differentiation for the computation of sparse Jacobians, , • Chaco -A graph partitioning package, • ESSL -IBM's math library for fast sparse direct LU factorization, • Euclid -parallel ILU(k) developed by David Hysom, accessed through the Hypre interface, • Hypre -the LLNL preconditioner library, • LUSOL -sparse LU factorization code (part of MINOS) developed by Michael Saunders, Systems
This manual describes the use of PETSc 2.0 for the numerical solution of partial differential equ... more This manual describes the use of PETSc 2.0 for the numerical solution of partial differential equations and related problems on high-performance computers. The Portable, Extensible Toolkit for Scientific computation (PETSc) is a suite of data structures and routines that provide the building blocks for the implementation of large-scale application codes on parallel (and serial) computers. PETSc 2.0 uses the MPI standard for all message-passing communication. PETSc includes an expanding suite of parallel linear and -nonlinear equation solvers that may be used in application codes ,written in Fortran, C, and C++. PETSc provides many of the mechanisms needed within parallel application codes, such as simple parallel matrix and vector assembly routines that allow the overlap of communication and computation. In addition, PETSc includes growing support for distributed arrays. The library is organized hierarchically, enabling users to employ the level of abstraction that is most appropriate for a particular problem. By using techniques of object-oriented programming, PETSc provides enormous flexibility for users. PETSc is a sophisticated set of software tools; as such, for some users it initially has a much steeper learning curve than a simple subroutine library. In particular, for individuals without some computer science background or experience programming in C, Pascal, or C++, it may require a large amount of time to take full advantage of the features that enable efficient software use. However, the power of the PETSc design and the algorithms it incorporates make the efficient implementation of many application codes much simpler than "rolling them" yourself. For many simple (or even relatively complicated) tasks a package such as Matlab is often the best tool; PETSc is not intended for the classes of problems for which effective Matlab code can be written. Since PETSc is still under development, small changes in usage and calling sequences of PETSc routines will continue to occur. Although keeping one's code up to date can be somewhat annoying, all PETSc users will be rewarded in the long run with a cleaner, better designed,
This manual describes the use of PETSc 2.0 for the numerical solution of partial differential equ... more This manual describes the use of PETSc 2.0 for the numerical solution of partial differential equations and related problems on high-performance computers. The Portable, Extensible Toolkit for Scientific computation (PETSc) is a suite of data structures and routines that provide the building blocks for the implementation of large-scale application codes on parallel (and serial) computers. PETSc 2.0 uses the MPI standard for all message-passing communication. PETSc includes an expanding suite of parallel linear and -nonlinear equation solvers that may be used in application codes ,written in Fortran, C, and C++. PETSc provides many of the mechanisms needed within parallel application codes, such as simple parallel matrix and vector assembly routines that allow the overlap of communication and computation. In addition, PETSc includes growing support for distributed arrays. The library is organized hierarchically, enabling users to employ the level of abstraction that is most appropriate for a particular problem. By using techniques of object-oriented programming, PETSc provides enormous flexibility for users. PETSc is a sophisticated set of software tools; as such, for some users it initially has a much steeper learning curve than a simple subroutine library. In particular, for individuals without some computer science background or experience programming in C, Pascal, or C++, it may require a large amount of time to take full advantage of the features that enable efficient software use. However, the power of the PETSc design and the algorithms it incorporates make the efficient implementation of many application codes much simpler than "rolling them" yourself. For many simple (or even relatively complicated) tasks a package such as Matlab is often the best tool; PETSc is not intended for the classes of problems for which effective Matlab code can be written. Since PETSc is still under development, small changes in usage and calling sequences of PETSc routines will continue to occur. Although keeping one's code up to date can be somewhat annoying, all PETSc users will be rewarded in the long run with a cleaner, better designed,
Proceedings in applied mathematics & mechanics, Dec 1, 2007
Zenodo (CERN European Organization for Nuclear Research), Apr 17, 2016
Uploads
Papers by Victor Eijkhout