Generic programming for high-performance scientific applications

Microprocessors and Microsystems, 1987

Programming multiprocessor parallel architectures is a complex task. This paper describes a Pascal-like scientific programming language, Blaze, designed to simplify this task. Blaze contains array arithmetic, "forall" loops, and APL-style accumulation operators, which allow natural expression of fine grained parallelism. It also employs an applicative or functional procedure invocation mechanism, which makes it easy for compilers to extract coarse grained parallelism using machine specifiC program restructuring. Thus Blaze should allow one to achieve highly parallel execution on multiprocessor architectures, while still providing the user with conceptually sequential control flow.

In this paper we focus on generating efficient parallel codes for solving scientific problems within large-scale performance-critical applications. More specifically, we present techniques for high-level symbolic optimization for automatic generation of efficient codes that numerically solve sets of computationally expensive right-hand sides (RHSs) of systems of partial differential equations (PDEs). These type of PDE problems can be found in application areas like numerical weather prediction, ocean circulation modeling, and climate modeling. To solve the PDEs describing the physical problem, high-performance software environments and computer platforms are required. The CTADEL application driver is a software environment for generating optimized codes for serial, vector, or parallel architectures from a high-level PDE problem description of a weather forecast model. The advantage of the high-level problem description is that all of the high-level information can be exploited for parallelization, restructuring, and optimization from the highest descriptive level to the low-level generated code.

2006

xingca,hpl]@simula.no Summary. This chapter aims to answer the following question: Can the high-level programming language Python be used to develop sufficiently efficient parallel solvers for partial differential equations (PDEs)? We divide our investigation into two aspects, namely (1) the achievable performance of a parallel program that extensively uses Python programming and its associated data structures, and (2) the Python implementation of generic software modules for parallelizing existing serial PDE solvers. First of all, numerical computations need to be based on the special array data structure of the Numerical Python package, either in pure Python or in mixed-language Python-C/C++ or Python/Fortran setting. To enable highperformance message passing in parallel Python software, we use the small add-on package pypar, which provides efficient Python wrappers to a subset of MPI routines. Using concrete numerical examples of solving wave-type equations, we will show that a mixed Python-C/Fortran implementation is able to provide fully comparable computational speed in comparison with a pure C or Fortran implementation. In particular, a serial legacy Fortran 77 code has been parallelized in a relatively straightforward manner and the resulting parallel Python program has a clean and simple structure.

The International Journal of High Performance Computing Applications, 2007

We describe the rationale behind the design of key features of Titanium—an explicitly parallel dialect of Java for high-performance scientific programming—and our experiences in building applications with the language. Specifically, we address Titanium's partitioned global address space model, single program multiple data parallelism support, multi-dimensional arrays and array-index calculus, memory management, immutable classes (class-like types that are value types rather than reference types), operator overloading, and generic programming. We provide an overview of the Titanium compiler implementation, covering various parallel analyses and optimizations, Titanium runtime technology and the GASNet network communication layer. We summarize results and lessons learned from implementing the NAS parallel benchmarks, elliptic and hyperbolic solvers using adaptive mesh refinement, and several applications of the immersed boundary method.

Lecture Notes in Computer Science, 2000

Charon is a library, callable from C and Fortran, that aids the conversion of structuredgrid legacy codes-such as those used in the numerical computation of fluid flows-into parallel, high-performance codes. Key are functions that define distributed arrays, that map between distributed and non-distributed arrays, and that allow easy specification of common communications on structured grids. The library is based on the widely accepted MPI message passing standard. We present an overview of the functionality of Charon, and some representative results. 1 Introduction A sign of the maturing of the field of parallel computing is the emergence of facilities that shield the programmer from low-level constructs such as message passing (MPI, PVM) and shared memory parallelization directives (P-Threads, OpenMP, etc.), and from parallel programming languages (High Performance Fortran, Split C, Linda, etc.). Such facilities include: 1) (semi-)automatic tools for parallelization of legacy codes (e.g. CAPTools [8], ADAPT [5], CAPO [g], SUIF compiler [6], SMS preprocessor [7], etc.), and 2) application libraries for the construction of parallel programs from scratch (KELP [4], OVERTURE [3], PETSc [2], Global Arrays [10], etc.). The Charon library described here offers an alternative to the above two approaches, namely a mechanism for incremental conversion of legacy codes into high-performance, scalable messagepassing programs. It does so without the need to resort to explicit parallel programming constructs. Charon is aimed at applications that involve structured discretization grids used for the solution of scientific computing problems. Specifically, it is designed to help parallelize algorithms that are not naturally data parallel-i.e., that contain complex data dependencies-which include almost all advanced flow solver methods in use at NASA Ames Research Center. Whi_e Charon provides strictly a set o£ user-callable functions (C and Fortran), it can nonetheless be used to https://ntrs.nasa.gov/search.jsp?R=20020043296 2020-06-07T10:50:16+00:00Z convert serial legacy codes into highly-tuned parallel applications. The crux of the library is that it enables the programmer to codify information about existing multi-dimensional arrays in legacy codes and map between these non-distributed arrays and newly defined truly distributed arrays at runtime. This allows the programmer to keep most of the serial code unchanged and only use distributed arrays in that part of the code of prime interest. This code section, which is parallelized using more functions from the Charon library, is gradually expanded, until the entire code is converted. The major benefit of incremental parallalization is that it is easy to ascertain consistency with the serial code. In addition, the user keeps careful control over the amount of data transferred between processes, which is important on high-latency distributed-memory machines 1. The usual steps that a programmer follows when parallelizing a code using Charon are as follows. First, define a distribution of the arrays in the program, based on a division of the grid(s) among all processors. Second, select a section of the code to be parallelized, and construct a so-called parallel bypass: map from the non-distributed (legacy code) array to the distributed array upon entry of the section, and back to the non-distributed array upon leaving it. Third, do the action parallelization work for the section, using more Charon functions. The remainder of this paper is structured as follows. In Section 2 we explain the library functions used to define and manipulate distributed arrays (distributions), including those that allow the mapping between non-distributed and distributed arrays. In Section 3 we describe the functions that can be used actually to parallelize an existing piece of code. Some examples of the use and performance of Charon are presented in Section 4. 2 Distributed arrays Parallelization of scientific computations using Charon is based on the concept of domain decomposition. Typically, one or more multi-dimensional grids are defined, and arrays-representing computational work-are associated with these grids. The grids are divided into nonoverlapping 1We will henceforthspeak of processors, even though processes is the more accurate term. pieces,which are assignedto the processors in the computation. The associated arrays are thus distributed as well. This process takes place in several steps, as illustrated in Figure 1, and as described below. First (Figure la), the logically rectangular discretization grid of a certain dirnensionality and extent is defined, using OliN_Create_grid. The purpose of this step is to establish a geometric framework for all arrays associated with the grid. It also attaches to the grid an MPI [11] communicator, which serves as the context and processor subspace within which all subsequent Charon-orchestrated communications will take place. Multiple coincident or non-coincident communicators may be used within one program, allowing the programmer to assign the same or different (sets of) processors to different grids in a multiple-grid computation. Second (Figure lb), tessellations of the domain (sections) are defined, based on the grid variable. The associated library call is CHN_Create_section. Sections contain a number of cutting planes (cuts) along each coordinate direction. The grid is thus carved into a number of cells, each of which contains a logically rectangular block of grid points. Whereas the programmer can specify any number of cuts and cut locations, it is often sufficient to make a single call to a high-level routine to define all the cuts belonging to a particular kind of domain decomposition. For example, defining a section with just a single cell (i.e. a non-divided grid with zero cuts) is accomplished with CHN_Set_solopartition_cuts. Using CHN_Set_unipartition_cuts divides the grid evenly into as many cells as there are processors in the communicator. Third (Figure lc), the cells created in a section have to be assigned to processors, resulting in a decomposition. The associated library call is CHN_Create_decomposition. The reason why the creation of section and decomposition are separated is to provide flexibility. For example, the programmer may want to divide a grid into ten slices for execution on a parallel computer, but may want to assign all these slices to the same processor for the purpose of debugging on a serial machine. As in the case of the creation of sections, the programmer can choose to assign each cell to a processor individually, or make a single call to a high-level routine. For example, the function CHN_Set_unipartition_owners assigns each of the cells in the unipartition section CHN_Create_Oeoomposition(y_cmp,y_se¢); CHN_Set_uniparlitio__owners(y_cmp);

1994

There is a need for compiler technology that, given the source program, will generate efficient parallel codes for different architectures with minimal user involvement. Parallel computation is becoming indispensable in solving large-scale problems in science and engineering. Yet, the use of parallel computation is limited by the high costs of developing the needed software.

Springer eBooks, 2015

Languages and Compilers for Parallel Computing, 1995

The development of efficient numerical programs and library routines for high-performance parallel computers is a complex task requiring not only an understanding of the algorithms to be implemented, but also detailed knowledge of the target machine and the software environment. In this paper, we describe a programming environment that can utilize such knowledge for the development of high-performance numerical programs and libraries. This environment uses an existing high-level array language (MATLAB) as source ...

The Architecture of Scientific Software, 2001

A distinguished feature of scientific computing is the necessity to design software abstractions for approximations. The approximations are themselves abstractions of mathematical models, and the mathematical models are also abstractions of the real world. In this paper, the relation between different mathematical abstraction levels and scientific computing software is discussed, in particular with respect to the simulation of partial differential equations. By applying software engineering practices already when the mathematical model is considered, a coordinate-free formulation is readily motivated. For partial differential equations, the continuos layer is identified as separate from the the coordinate-free layer. By mapping these layers into different software modules, numerical discretization carried out in the continuous layer is cleanly separated from the coordinate-free layer. This separation of concerns increases modularity because reusability is promoted. It is therefore concluded that the continuous and coordinate-free abstraction layers provide a solid foundation for software that simulates partial differential equations.