Large-scale parallel data clustering

Lecture Notes in Computer Science, 2003

Clustering can be defined as the process of partitioning a set of patterns into disjoint and homogeneous meaningful groups (clusters). There is a growing need for parallel algorithms in this field since databases of huge size are common nowadays. This paper presents a parallel version of a recently proposed algorithm that has the ability to scale very well in parallel environments.

Ingénierie des systèmes d information, 2021

K-means++ is the clustering algorithm that is created to improve the process of getting initial clusters in the K-means algorithm. The k-means++ algorithm selects initial k-centroids arbitrarily dependent on a probability that is proportional to each data-point distance to the existing centroids. The most noteworthy problem of this algorithm is when running happens in sequential mode, as this reduces the speed of clustering. In this paper, we develop a new parallel k-means++ algorithm using the graphics processing units (GPU) where the Open Computing Language (OpenCL) platform is used as the programming environment to perform the data assignment phase in parallel while the Streaming SIMD Extension (SSE) technology is used to perform the initialization step to select the initial centroids in parallel on CPU. The focus is on optimizations directly targeted to this architecture to exploit the most of the available computing capabilities. Our objective is to minimize runtime while keepi...

The k-means clustering algorithm and neural network batch training becomes computationally intensive when the manipulated data is large. One way to reduce the computational demand of such techniques is to execute them in a concurrent manner. Unfortunately, the effort required to implement these techniques in a distributed computing environment remains daunting. Much of the work takes place when partitioning and distributing workloads over processors in the distributed computing environment. To alleviate this task, we present a data parallel interface called Distributed Data Partitioning Interface (DDPI). Its simple interface permits parallel implementation of k-means type clustering algorithms and neural network batch training even by users with little understanding of parallel computing technicalities. In this work we demonstrate that it is possible to achieve near ideal speedups when k-means and k-harmonic means clustering algorithms and multilayer perceptron batch training are parallelized with DDPI.

2008

ABSTRACT Technology advances suggest that the data deluge, network bandwidth and computers performance will continue their exponential increase. Computers will exhibit 64-128 cores in some 5 years. Consequences include a growing importance of data mining and data analysis capabilities that need to perform well on both parallel and distributed Grid systems. We discuss a class of such algorithms important in chemoinformatics, bioinformatics and demographic studies.

2000

This paper describes the design and implementation on MIMD parallel machines of P-AutoClass, a parallel version of the AutoClass system based upon the Bayesian method for determining optimal classes in large datasets. The P-AutoClass implementation divides the clustering task among the processors of a multicomputer so that they work on their own partition and exchange their intermediate results.

Journal of the American Society for Information Science and Technology, 2007

A distributed memory parallel version of the group average Hierarchical Agglomerative Clustering algorithm is proposed to enable scaling the document clustering problem to large collections. Using standard message passing operations reduces interprocess communication while maintaining efficient load balancing. In a series of experiments using a subset of a standard TREC test collection, our parallel hierarchical clustering algorithm is shown to be scalable in terms of processors efficiently used and the collection size. Results show that our algorithm performs close to the expected O(n 2 /p) time on p processors, rather than the worst-case O(n 3 /p) time. Furthermore, the O(n 2 /p) memory complexity per node allows larger collections to be clustered as the number of nodes increases. While partitioning algorithms such as k-means are trivially parallelizable, our results confirm those of other studies showing that hierarchical algorithms produce significantly tighter clusters in the document clustering task. Finally, we show how our parallel hierarchical agglomerative clustering algorithm can be used as the clustering subroutine for a parallel version of the Buckshot algorithm to cluster the complete TREC collection at near theoretical runtime expectations.

An optimal data partitioning in parallel/distributed implementation of clustering algorithms is a necessary computation as it ensures independent task completion, fair distribution, less number of affected points and better & faster merging. Though partitioning using Kd-Tree is being conventionally used in academia, it suffers from performance drenches and bias (non equal distribution) as dimensionality of data increases and hence is not suitable for practical use in industry where dimensionality can be of order of 100's to 1000's. To address these issues we propose two new partitioning techniques using existing mathematical models & study their feasibility, performance (bias and partitioning speed) & possible variants in choosing initial seeds. First method uses an n-dimensional hashed grid based approach which is based on mapping the points in space to a set of cubes which hashes the points. Second method uses a tree of voronoi planes where each plane corresponds to a partition. We found that grid based approach was computationally impractical, while using a tree of voronoi planes (using scalable K-Means++ initial seeds) drastically outperformed the Kd-tree tree method as dimensionality increased.

There are three kinds of clustering approaches: partitioning clustering, hierarchical clustering, and hybrid clustering in terms of cluster structures. Hierarchical clustering is more flexible than partitioning clustering but very expensive for large data sets. Hybrid clustering combines the features of hierarchical and partitioning clustering. In this paper, we propose a parallel hybrid clustering method which is implemented using MPI (message passing interface). We run (ran) the parallel program on different numbers of computer nodes/machines and compared the results with the sequential program. Experiments show that our parallel approach is more efficient than the sequential clustering and produces better results when the number of machines is large.

Journal of Convergence Information Technology, 2008

Distributed processing today is a largely advantageous technology of bridging together a system of multiple computers and processor systems in running applications. The concept of Distributed processing has allowed time cutting and therefore reduction in costs. Using this, we aim to address clustering techniques in developing new method for further reduction in time and costs. The problem of clustering huge amount of data is a very time consuming operation. So by applying parallel and distributed approach, we can minimize the total time necessary for clustering the data. In this research, a parallel and distributed version of k-means clustering algorithm is proposed. The proposed algorithm will be implemented using Matlab, and will be tested with large synthetic data sets.

Mathematics and Visualization, 2004

Clustering is a classical data analysis technique that is applied to a wide range of applications in the sciences and engineering. For very large data sets, the performance of a clustering algorithm becomes critical. Although clustering has been thoroughly studied over the last decades, little has been done on utilizing modern multi-processor machines to accelerate the analysis process. We propose a scalable clustering technique that benefits from existing parallel computers and networks of workstations. It supports the creation of multiresolution representations for very large geometric data sets. The output of the clustering process can be used for interactive data exploration, useful for view-dependent rendering, user-guided refinement, and progressive transmission.