High Performance Regional Ocean Modeling with GPU Acceleration

ArXiv, 2019

In this work, we take a modern high-resolution finite-volume scheme for solving the rotational shallow-water equations and extend it with features required to run real-world ocean simulations. Our contributions include a spatially varying north vector and Coriolis term required for large scale domains, moving wet-dry fronts, a static land mask, bottom shear stress, wind forcing, boundary conditions for nesting in a global model, and an efficient model reformulation that makes it well-suited for massively parallel implementations. Our model order is verified using a grid convergence test, and we show numerical experiments using three different sections along the coast of Norway based on data originating from operational forecasts run at the Norwegian Meteorological Institute. Our simulation framework shows perfect weak scaling on a modern P100 GPU, and is capable of providing tidal wave forecasts that are very close to the operational model at a fraction of the cost. All source code ...

2016

The Parallel Ocean Program (POP) is used in many strongly eddying ocean circulation simulations. Ideally it would be desirable to be able to do thousand-yearlong simulations, but the current performance of POP prohibits these types of simulations. In this work, using a new distributed computing approach, two methods to improve the performance of POP are presented. The first is a blockpartitioning scheme for the optimization of the load balancing of POP such that it can be run efficiently in a multiplatform setting. The second is the implementation of part of the POP model code on graphics processing units (GPUs). We show that the combination of both innovations also leads to a substantial performance increase when running POP simultaneously over multiple computational platforms.

Parallel Computing, 1995

A two-pronged e ort to convert a recently developed ocean circulation model written in Fortran-77 for execution on massively parallel computers is described. A data-parallel version has been developed for the CM-5 manufactured by Thinking Machines, Inc., while a message-passing version has been developed for the Cray T3D. Since the time di erentiation scheme in the ocean model is fully explicit{meaning that all equations are hyperbolic{, satisfactory machine utilization has been achieved without major changes to the original algorithms. On a per-processor basis, the code has been found to execute 2.3 to 4.4 times faster on the T3D than on the CM-5.

1994

We describe new parallelization techniques applied to a highly parallel ocean circulation application code – the Miami Isopycnic Ocean Coordinate Model or MICOM. We compare three parallel architectures executing MICOM: vector, massively parallel, and a multiprocessor workstation. Results from a high resolution 0.08 MICOM North Atlantic basin calculation on the Cray T3D are described briefly. This high-resolution calculation gives qualitatively different results than lower resolution runs implying that a resolution threshold must be crossed to retain fidelity to observed ocean currents.

The numerical representation of the equation of state (EOS) in ocean models is reevaluated in terms of its computational efficiency. Ways to speedup the EOS as well as basic codes in ocean models are discussed and demonstrated. It is shown that even a simple rearrangement of a calculation order can significantly speedup the codes currently used in several community terrain-following ocean models. Using a new EOS with particular fit to data that takes into account the computational cost, can speedup the code even further. Various EOS codes were tested on different computer platforms, demonstrating the potential of speeding up codes by as much as 2-3 times on a supercomputer and up to 6-16 times on laptops and personal computers.

During the course of this project dCSE Fluidity-ICOM has been transformed from a code that was primarily used on institution level clusters with typically 64 tasks used per simulation into a highly performing scalable code which can be run efficiently on 4096 cores of the current HECToR hardware (Cray XT4 Phase2a). Fluidity-ICOM has been parallelised with MPI and optimised for HECToR alongside continual in-depth performance analysis.

International Journal of High Performance Computing Applications, 2013

This paper focuses on the parallelization of an ocean model applying current multicore processor-based cluster architectures to an irregular computational mesh. The aim is to maximize the efficiency of the computational resources used. To make the best use of the resources offered by these architectures, this parallelization has been addressed at all the hardware levels of modern supercomputers: firstly, exploiting the internal parallelism of the CPU through vectorization; secondly, taking advantage of the multiple cores of each node using OpenMP; and finally, using the cluster nodes to distribute the computational mesh, using MPI for communication within the nodes. The speedup obtained with each parallelization technique as well as the combined overall speedup have been measured for the western Mediterranean Sea for different cluster configurations, achieving a speedup factor of 73.3 using 256 processors. The results also show the efficiency achieved in the different cluster nodes and the advantages obtained by combining OpenMP and MPI versus using only OpenMP or MPI. Finally, the scalability of the model has been analysed by examining computation and communication times as well as the communication and synchronization overhead due to parallelization.

2002

Our long-term goal is to perform an eddy-resolving global ocean simulation using the Parallel Ocean Program (POP) that will provide a high fidelity ocean model state for use as an initial condition in a global ocean/atmosphere/ice prediction system. In addition, the global model state may be used to specify boundary conditions for regional circulation models.

Journal of Parallel and Distributed Computing, 1998

Two parallel implementations of a state-of-the-art ocean model are described and analyzed: one is written in the implicitly parallel language Id for the Monsoon multithreaded dataflow architecture, and the other in data parallel CM Fortran for the CM-5. The multithreaded programming model is inherently more expressive than the data parallel model. One goal of this study is to understand what, if any, are the performance penalties of multithreaded execution when implementing a program that is well suited for data parallel execution. To avoid technology and machine configuration issues, the two implementations are compared in terms of overhead cycles per required floating point operation. When simulating flows in complex geometries typical of ocean basins, the data parallel model only remains efficient if redundant computations are performed over land. The generality of the Id programming model allows one to easily and transparently implement a parallel code that computes only in the ocean. When simulating ocean basins with complex and irregular geometry the normalised performance on Monsoon is comparable with that of the CM-5. For more regular geometries, that map well to the computational domain, the data-parallel approach proves to be a better match. We conclude by examining the extent to which clusters of mainstream symmetric multiprocessor (SMP) systems offer a scientific computing environment which can capitalize on and combine the strengths of the two paradigms.

Ocean Dynamics, 2014