Analysis of Threading Libraries for High Performance Computing

2008 IEEE International Symposium on Parallel and Distributed Processing, 2008

Large scale hardware-supported multithreading, an attractive means of increasing computational power, benefits significantly from low per-thread costs. Hardware support for lightweight threads is a developing area of research. Each architecture with such support provides a unique interface, hindering development for them and comparisons between them. A portable abstraction that provides basic lightweight thread control and synchronization primitives is needed. Such an abstraction would assist in exploring both the architectural needs of large scale threading and the semantic power of existing languages. Managing thread resources is a problem that must be addressed if massive parallelism is to be popularized. The qthread abstraction enables development of large-scale multithreading applications on commodity architectures. This paper introduces the qthread API and its Unix implementation, discusses resource management, and presents performance results from the HPCCG benchmark.

Journal of Parallel and Distributed …, 1996

Lightweight threads are becoming increasingly useful in supporting parallelism and asynchronous events in applications and language implementations. Traditionally, lightweight threads are supported only within the single address space of a process, or in shared memory environments with multiple processes. We introduce and describe the design of Chant, a runtime system supporting lightweight threads in a distributed memory environment. In addition to communication between any two threads in the system, Chant provides support for remote service requests, remote thread operations, and collective communication between thread groups called ropes. Chant provides the rst implementation of lightweight threads for a distributed memory platform whose design incorporates existing standards for lightweight threads and interprocess communication. This paper details the issues that arise in extending a standard threads package to support distributed memory execution, and the solutions that are provided by the Chant system.

2002

Goal of this paper is the efficient and portable implementation of the Nanothreads Programming Model (NPM), a dependence-driven two-level threads model for shared-memory multiprocessors, on top of the POSIX Threads API. Our general approach is to combine portability with efficiency by implementing the machine dependent components of the runtime system based on POSIX threads and defining an appropriate API that any underlying non-preemptive user-level threads package should export in order to get integrated with it. We implement such a portable lightweight threads package based on POSIX calls, and then we replace this package with POSIX threads. On M:N implementation models, we exploit their lightweight process scope POSIX threads. Our approach makes the implementation of NPM fully portable while, on the other hand, provides a significant extension of POSIX threads. The portability of our work allows its experimental evaluation on several operating systems and hardware platforms.

2020

OpenMP has been the de facto standard for single node parallelism for more than a decade. Recently, asynchronous many-task runtime (AMT) systems have increased in popularity as a new programming paradigm for high performance computing applications. One of the major challenges of this new paradigm is the incompatibility of the OpenMP thread model and other AMTs. Highly optimized OpenMP-based libraries do not perform well when coupled with AMTs because the threading of both libraries will compete for resources. This paper is a follow-up paper on the fundamental implementation of hpxMP, an implementation of the OpenMP standard which utilizes the C++ standard library for Parallelism and Concurrency (HPX) to schedule and manage tasks. In this paper, we present the implementation of task features, e.g. taskgroup, task depend, and task_reduction, of the OpenMP 5.0 standard and optimization of the #pragma omp parallel for pragma. We use the daxpy benchmark, the Barcelona OpenMP Tasks Suite,...

2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS), 2010

The now commonplace multi-core chips have introduced, by design, a deep hierarchy of memory and cache banks within parallel computers as a tradeoff between the user friendliness of shared memory on the one side, and memory access scalability and efficiency on the other side. However, to get high performance out of such machines requires a dynamic mapping of application tasks and data onto the underlying architecture. Moreover, depending on the application behavior, this mapping should favor cache affinity, memory bandwidth, computation synchrony, or a combination of these. The great challenge is then to perform this hardware-dependent mapping in a portable, abstract way.

2000

This paper presents a set of proposals for the OpenMP shared-memory programming model oriented towards the definition of thread groups in the framework of nested parallelism. The paper also describes the additional functionalities required in the runtime library supporting the parallel execution. The extensions have been implemented in the OpenMP NanosCompiler and evaluated in a set of real applications and benchmarks. In this paper we present experimental results for one of these applications.

OpenMP provides a portable programming interface on shared memory multiprocessors (SMPs). The set of features in the current OpenMP specification provides essential functionality that was selected mostly from existing shared-memory parallel application programming interfaces (APIs). Although this interface has proven successful for small SMPs, it requires greater flexibility in light of the steadily growing size of individual SMPs and the recent advent of multithreaded chips. In this paper, we introduce the syntax and semantics of a proposed OpenMP extension for facilitating the expression of worksharing on the emerging chip multithreading architectures. We also describe our implementation in the OpenUH reference OpenMP compiler and the runtime library. We then evaluate the new feature using a kernel of seismic data processing application.

2021 29th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), 2021

Enabling efficient fine-grained task parallelism is a significant challenge for hardware platforms with increasingly many cores. Existing techniques do not scale to hundreds of threads due to the high cost of synchronization in concurrent data structures. To overcome these limitations we present XQueue, a novel lock-less concurrent queuing system with relaxed ordering semantics that is geared towards realizing scalability up to hundreds of concurrent threads. We demonstrate the scalability of XQueue using microbenchmarks and show that XQueue can deliver concurrent operations with latencies as low as 110 cycles at scales of up to 192 cores (up to 6900× improvement compared to traditional synchronization mechanisms) across our diverse hardware, including x86, ARM, and Power9. The reduced latency allows XQueue to provide orders of magnitude (3300×) better throughput that existing techniques. To evaluate the real-world benefits of XQueue, we integrated XQueue with LLVM OpenMP and evaluated five unmodified benchmarks from the Barcelona OpenMP Task Suite (BOTS) as well as a graph traversal benchmark from the GAP benchmark suite. We compared the XQueue-enabled LLVM OpenMP implementation with the native LLVM and GNU OpenMP versions. Using fine-grained task workloads, XQueue can deliver 4× to 6× speedup compared to native GNU OpenMP and LLVM OpenMP in many cases, with speedups as high as 116× in some cases.

OpenMP: Enabling Massive Node-Level Parallelism, 2021

OpenMP, as the de-facto standard programming model in symmetric multiprocessing for HPC, has seen its performance boosted continuously by the community, either through implementation enhancements or specification augmentations. Furthermore, the language has evolved from a prescriptive nature, as defined by the thread-centric model, to a descriptive behavior, as defined by the task-centric model. However, the overhead related to the orchestration of tasks is still relatively high. Applications exploiting very fine-grained parallelism and systems with a large number of cores available might fail on scaling. In this work, we propose to include the concept of Task Dependency Graph (TDG) in the specification by introducing a new clause, named taskgraph, attached to task or target directives. By design, the TDG allows alleviating the overhead associated with the OpenMP tasking model, and it also facilitates linking OpenMP with other programming models that support task parallelism. According to our experiments, a GCC implementation of the taskgraph is able to significantly reduce the execution time of fine-grained task applications and increase their scalability with regard to the number of threads.

IEEE Transactions on Computers, 1989

Abstmct-Threads ("lightweight" processes) have become a common element of new languages and operating systems. This paper examines the performance implications of several data structure and algorithm alternatives for thread management in shared-memory multiprocessors. Both experimental measurements and analytical model projections are presented.