Em Clustering

Nowadays, financial data analysis is becoming increasingly important in the business market. As companies collect more and more data from daily operations, they expect to extract useful knowledge from existing collected data to help make reasonable decisions for new customer requests, e.g. user credit category, confidence of expected return, etc. Banking and financial institutes have applied different data mining techniques to enhance their business performance. Among these techniques, clustering has been considered as a significant method to capture the natural structure of data. However, there are not many studies on clustering approaches for financial data analysis. In this paper, we evaluate different clustering algorithms for analysing different financial datasets varied from time series to transactions. We also discuss the advantages and disadvantages of each method to enhance the understanding of inner structure of financial datasets as well as the capability of each clustering method in this context. ODAY, we have a deluge of financial datasets. Faster and cheaper storage technology allows us to store ever-greater amounts of data. Due to the large sizes of the data sources it is not possible for a human analyst to come up with interesting information (or patterns) that will help in the decision making process. Global competitions, dynamic markets, and rapid development in the information and communication technologies are some of the major challenges in today's financial industry. For instance, financial institutions are in constant needs for more data analysis, which is becoming more very large and complex. As the amount of data available is constantly increasing, our ability to process it becomes more and more difficult. Efficient discovery of useful knowledge from these datasets is therefore becoming a challenge and a massive economic need. On the other hand, data mining (DM) is the process of extracting useful, often previously unknown information, so-called knowledge, from large datasets (databases or data). This mined knowledge can be used for various applications such as market analysis, fraud detection, customer retention, etc. Recently, DM has proven to be very effective and N-A.

A performance evaluation of four different clustering techniques was carried out based on segmenting consumer by product type and by product usage in the research. Cobweb, DBSCAN, EM and k-means algorithms were evaluated based on the computational time, accuracy of the result produced and the purity of the result produced. The experiment was performed using WEKA as a data mining tool. The performance evaluation of the four techniques showed that K-means outperformed others in all considered evaluation measure while the EM technique was the second best in terms of accuracy and purity, outperforming the other two. DBSCAN technique was the 3 rd best of the selected algorithms even as its computational time is shorter than that of EM while the fourth best performing calculation has been believed to be the Spider web calculation as respects to immaculateness, exactness and computational time.

Classification of KDD intrusion dataset is proposed along with noise reduction, clustering and feature selection. DBSCAN algorithm has been applied to reduce noise present in KDD dataset. After noise removal genetic search approach is utilize to pick relevant feature. K-Means clustering method is utilized to cluster the dataset and resultant dataset is tested by decision tree based Adaboost classifier. It is examined that suggested approaches gives 98.822% accuracy. A comparative analysis performed between proposed methods and KMSVM (Simple K-mean with SVM classification) and it is observed that proposed method gives better results. Keywords— Intrusion Detection System, Density based algorithm, K-Means Clustering, Decision Tree J48, AdaBoost

Clustering is a data mining problem of dividing documents into groups, such that documents in one group are more similar than those in other groups. The aim of this study is to propose a framework for comparing the accuracy of clustering algorithms. The study applies qualitative research through document analysis to review previous clustering algorithms' comparisons so as to obtain the issues/problems with such previous comparisons. We then deduce appropriate comparisons framework that addresses the problems. The study obtained the following comparison issues: Nature of comparison, nature of data, size of data, source of data, evaluation metrics, and parameter settings. Consequently, the study proposed the rules, formulae, and procedures needed to be used in a comparison. It is recommended that applying this framework will ensure that such evaluations and comparisons are done using formal procedures that will yield dependable results. This study suggests a further study to be done to apply this framework and do a comprehensive comparison of some clustering algorithms.

Clustering is a data mining problem of dividing documents into groups, such that documents in one group are more similar than those in other groups. The aim of this study is to propose a framework for comparing the accuracy of clustering algorithms. The study applies qualitative research through document analysis to review previous clustering algorithms' comparisons so as to obtain the issues/problems with such previous comparisons. We then deduce appropriate comparisons framework that addresses the problems. The study obtained the following comparison issues: Nature of comparison, nature of data, size of data, source of data, evaluation metrics, and parameter settings. Consequently, the study proposed the rules, formulae, and procedures needed to be used in a comparison. It is recommended that applying this framework will ensure that such evaluations and comparisons are done using formal procedures that will yield dependable results. This study suggests a further study to be done to apply this framework and do a comprehensive comparison of some clustering algorithms.

An automatic method to combine several local surrogate models is presented. This method is intended to build accurate and smooth approximation of discontinuous functions that are to be used in structural optimization problems. It strongly relies on the Expectation−Maximization (EM) algorithm for Gaussian mixture models (GMM). To the end of regression, the inputs are clustered together with their output values by means of parameter estimation of the joint distribution. A local expert is then built (linear, quadratic, artificial neural network, moving least squares) on each cluster. Lastly, the local experts are combined using the Gaussian mixture model parameters found by the EM algorithm to obtain a global model. This method is tested over both mathematical test cases and an engineering optimization problem from aeronautics and is found to improve the accuracy of the approximation.

Semi-supervised data classification is one of significant field of study in machine learning and data mining since it deals with datasets which consists both a few labeled and many unlabeled data. The researchers have interest in this field because in real life most of the datasets have this feature. In this paper we suggest a collective method for solving semi-supervised data classification problems. Examples in R 1 presented and solved to gain a clear understanding. For comparison between state of art methods, well-known machine learning tool WEKA is used. Experiments are made on real-world datasets provided in UCI dataset repository. Results are shown in tables in terms of testing accuracies by use of ten fold cross validation. Yarı-gözetimli veri sınıflandırma, makine öğrenme ve veri madenciliğinde önemli bir çalışma alanıdır çünkü az sayıda etiketli ve çok sayıda etiketsiz veri içeren veri kümeleri ile ilgilenmektedir. Gerçek hayat veri kümelerinin çoğu bu özelliği taşıdığından birçok araştırmacı bu alana ilgi duymaktadır. Bu makalede yarı-gözetimli veri sınıflandırma problemlerinin çözümü için kolektif bir yöntem önerilmiştir. Konuyu daha iyi anlamak için R 1 de tanımlı veri kümeleri oluşturup önerilen algoritmalar bu veri kümelerine uygulanmıştır. Gelişkin tekniklerle karşılaştırma yapmak için en iyi bilinen WEKA makine öğrenme programı kullanılmıştır. Çalışmalar UCI veri kümesi deposunda bulunan gerçek hayat veri kümeleri üzerinde uygulanmıştır. 10 katlı çapraz geçerlilik ölçütü kullanılarak elde edilen değerlendirme sonuçları tablolarda sunulmuştur.

Detection of abnormal events in a traffic scene is a highly challenging task due to vast field of view, continuous stream of video data, various object interactions and complex events in Video Surveillance. Hence, this research proposes novel schemes using machine learning approach to detect abnormal events such as illegal U-turn, presence of pedestrian in driving region, wrong side driving and frequent lane change. Recently, Density Based Spatial Clustering of Applications with Noise (DBSCAN) is a popular method that has been used for clustering the trajectory datasets. The existing Density Based Clustering approach used for Abnormal detection in traffic scene uses random selection of cluster radius (Eps) and minimum points (minpts) needed to form a cluster. This random selection is time consuming and inefficient clustering results in accuracy reduction in abnormal detection. So, Adaptive Density based Spatial Clustering of Applications with Noise (ADBSCAN) is proposed for the dete...

Anomaly detection concerns identifying anomalous observations or patterns that are a deviation from the dataset's expected behaviour. The detection of anomalies has significant and practical applications in several industrial domains such as public health, finance, Information Technology (IT), security, medical, energy, and climate studies. Density-Based Spatial Clustering of Applications with Noise (DBSCAN) Algorithm is a density-based clustering algorithm with the capability of identifying anomalous data. In this paper, a modified DBSCAN algorithm is proposed for anomaly detection in time-series data with seasonality. For experimental evaluation, a monthly temperature dataset was employed and the analysis set forth the advantages of the modified DBSCAN over the standard DBSCAN algorithm for the seasonal datasets. From the result analysis, we may conclude that DBSCAN is used for finding the anomalies in a dataset but fails to find local anomalies in seasonal data. The proposed ...

Human movement anomalies in indoor spaces commonly involve urgent situations, such as security threats, accidents, and fires. This paper proposes a two-phase framework for detecting indoor human trajectory anomalies based on density-based spatial clustering of applications with noise (DBSCAN). The first phase of the framework groups datasets into clusters. In the second phase, the abnormality of a new trajectory is checked. A new metric called the longest common sub-sequence using indoor walking distance and semantic label (LCSS_IS) is proposed to calculate the similarity between trajectories, extending from the longest common sub-sequence (LCSS). Moreover, a DBSCAN cluster validity index (DCVI) is proposed to improve the trajectory clustering performance. The DCVI is used to choose the epsilon parameter for DBSCAN. The proposed method is evaluated using two real trajectory datasets: MIT Badge and sCREEN. The experimental results show that the proposed method effectively detects hum...

An automatic method to combine several local surrogate models is presented. This method is intended to build accurate and smooth approximation of discontinuous functions that are to be used in structural optimization problems. It strongly relies on the Expectation−Maximization (EM) algorithm for Gaussian mixture models (GMM). To the end of regression, the inputs are clustered together with their output values by means of parameter estimation of the joint distribution. A local expert is then built (linear, quadratic, artificial neural network, moving least squares) on each cluster. Lastly, the local experts are combined using the Gaussian mixture model parameters found by the EM algorithm to obtain a global model. This method is tested over both mathematical test cases and an engineering optimization problem from aeronautics and is found to improve the accuracy of the approximation.

Clustering is a data mining problem of dividing documents into groups, such that documents in one group are more similar than those in other groups. The aim of this study is to propose a framework for comparing the accuracy of clustering algorithms. The study applies qualitative research through document analysis to review previous clustering algorithms' comparisons so as to obtain the issues/problems with such previous comparisons. We then deduce appropriate comparisons framework that addresses the problems. The study obtained the following comparison issues: Nature of comparison, nature of data, size of data, source of data, evaluation metrics, and parameter settings. Consequently, the study proposed the rules, formulae, and procedures needed to be used in a comparison. It is recommended that applying this framework will ensure that such evaluations and comparisons are done using formal procedures that will yield dependable results. This study suggests a further study to be done to apply this framework and do a comprehensive comparison of some clustering algorithms.

One way for enterprises to be successful in today's challenging market is to be agile and be flexible to handle market changes. Using a conceptual and operational framework for improving the enterprise and keeping their desired situation, is always required. In this paper, a service oriented decision support system-based framework is proposed. The framework seeks for service oriented architecture (SOA) governance and suggests the initial architecture of the enterprise to support agility and optimality. Also, for the stability purpose a structure including service platforms, analyzers and decision support systems are employed to analyze the enterprise and make better decisions.

One way for enterprises to be successful in today's challenging market is to be agile and be flexible to handle market changes. Using a conceptual and operational framework for improving the enterprise and keeping their desired situation, is always required. In this paper, a service oriented decision support system-based framework is proposed. The framework seeks for service oriented architecture (SOA) governance and suggests the initial architecture of the enterprise to support agility and optimality. Also, for the stability purpose a structure including service platforms, analyzers and decision support systems are employed to analyze the enterprise and make better decisions.

Data mining is a method to mine valuable hidden knowledge, patterns and associations from massive and sparse datasets. This process proceeds through various techniques e.g. classification, clustering and association etc. Clustering is an important data mining technique which group similar data items together in a group. In this study comparison is performed with six different clustering techniques using six different datasets. Comparison was performed on the basis of different evaluation parameters. By overall results it is concluded that k-Mean algorithm is best, simplest, produced quality clusters and has high performance amongst all other five algorithms. Performance of EM algorithm is worst amongst all other five algorithms as it took more time to produce inaccurate results. Hierarchical algorithm is best on small datasets but on huge datasets it took more time. Performance of density based Clustered and Canopy algorithm is almost same with slight difference in results. We also compared our study results with existing results and proved that proposed results are quite reasonable and accurate. Our research analysis and results make better understanding for cluster researcher to improve existing techniques and also to analyze more techniques and to propose a new clustering technique.

The term internet of things is a buzz word these days and as per Google survey conducted recently, it has even dominated the buzz word big data predominantly. However, IoT area is still not matured and is throwing light on lot of research issues towards the data mining researchers. Security in IoT throws several challenges because of limited resources. In this context, IoT gains importance once again from data miners towards anomaly mining or intrusion detection. Intrusion detection is classified as NPclass in the literature even today. Algorithms addressing privacy and security issues in IoT must consider the complexities involved and hence require re-attention from all researchers. One more problem faced when judging for intrusion is the use of high dimensionality, classifier choice, and distance measure. For example, the traditional distance measure, such as Euclidean misjudges the similarity. In this paper, the objective is to design a fuzzy membership function to address both dimensionality and anomaly mining so as reduce the computational complexity and increase computational accuracies of classifier algorithms. We validate the proposed measure using several experimentations on NSL-KDD and Darpa datasets using KNN, J48 and CANN using Gaussian measure. Improved accuracies of classifiers on U2R and R2L attacks have been recorded in the experimental results obtained for experiments conducted.

The term internet of things is a buzz word these days and as per Google survey conducted recently, it has even dominated the buzz word big data predominantly. However, IoT area is still not matured and is throwing light on lot of research issues towards the data mining researchers. Security in IoT throws several challenges because of limited resources. In this context, IoT gains importance once again from data miners towards anomaly mining or intrusion detection. Intrusion detection is classified as NPclass in the literature even today. Algorithms addressing privacy and security issues in IoT must consider the complexities involved and hence require re-attention from all researchers. One more problem faced when judging for intrusion is the use of high dimensionality, classifier choice, and distance measure. For example, the traditional distance measure, such as Euclidean misjudges the similarity. In this paper, the objective is to design a fuzzy membership function to address both dimensionality and anomaly mining so as reduce the computational complexity and increase computational accuracies of classifier algorithms. We validate the proposed measure using several experimentations on NSL-KDD and Darpa datasets using KNN, J48 and CANN using Gaussian measure. Improved accuracies of classifiers on U2R and R2L attacks have been recorded in the experimental results obtained for experiments conducted.

Reliable network traffic classification is essential to management and security tasks. Therefore, it is beneficial to analyze and improve existing techniques. Some of the most traditional methodologies for traffic classification are based on port number and packet payload, each of which presents an increasing set of problems. Port number-based classification techniques suffer from the misuse of port numbers and tunneling. This is primarily due to their reliance on the proper use of IANA (Internet Assigned Numbers Authority) assigned numbers. On the other hand, packet payload-based classification has difficulty dealing with encrypted data and legal restrictions to accessing user data. Flow feature-based classification canovercome these challenges by creating profiles based on the traffic patterns of applications. Furthermore, machine learning techniques have shown to be a good match for traffic classification. Thus, the goal of this paper is to explore the combination of these fields...

Semi-supervised data classification is one of significant field of study in machine learning and data mining since it deals with datasets which consists both a few labeled and many unlabeled data. The researchers have interest in this field because in real life most of the datasets have this feature. In this paper we suggest a collective method for solving semi-supervised data classification problems. Examples in R 1 presented and solved to gain a clear understanding. For comparison between state of art methods, well-known machine learning tool WEKA is used. Experiments are made on real-world datasets provided in UCI dataset repository. Results are shown in tables in terms of testing accuracies by use of ten fold cross validation. Yarı-gözetimli veri sınıflandırma, makine öğrenme ve veri madenciliğinde önemli bir çalışma alanıdır çünkü az sayıda etiketli ve çok sayıda etiketsiz veri içeren veri kümeleri ile ilgilenmektedir. Gerçek hayat veri kümelerinin çoğu bu özelliği taşıdığından birçok araştırmacı bu alana ilgi duymaktadır. Bu makalede yarı-gözetimli veri sınıflandırma problemlerinin çözümü için kolektif bir yöntem önerilmiştir. Konuyu daha iyi anlamak için R 1 de tanımlı veri kümeleri oluşturup önerilen algoritmalar bu veri kümelerine uygulanmıştır. Gelişkin tekniklerle karşılaştırma yapmak için en iyi bilinen WEKA makine öğrenme programı kullanılmıştır. Çalışmalar UCI veri kümesi deposunda bulunan gerçek hayat veri kümeleri üzerinde uygulanmıştır. 10 katlı çapraz geçerlilik ölçütü kullanılarak elde edilen değerlendirme sonuçları tablolarda sunulmuştur.

Clustering is a data mining problem of dividing documents into groups, such that documents in one group are more similar than those in other groups. The aim of this study is to propose a framework for comparing the accuracy of clustering algorithms. The study applies qualitative research through document analysis to review previous clustering algorithms' comparisons so as to obtain the issues/problems with such previous comparisons. We then deduce appropriate comparisons framework that addresses the problems. The study obtained the following comparison issues: Nature of comparison, nature of data, size of data, source of data, evaluation metrics, and parameter settings. Consequently, the study proposed the rules, formulae, and procedures needed to be used in a comparison. It is recommended that applying this framework will ensure that such evaluations and comparisons are done using formal procedures that will yield dependable results. This study suggests a further study to be done to apply this framework and do a comprehensive comparison of some clustering algorithms.

Anomaly detection concerns identifying anomalous observations or patterns that are a deviation from the dataset's expected behaviour. The detection of anomalies has significant and practical applications in several industrial domains such as public health, finance, Information Technology (IT), security, medical, energy, and climate studies. Density-Based Spatial Clustering of Applications with Noise (DBSCAN) Algorithm is a density-based clustering algorithm with the capability of identifying anomalous data. In this paper, a modified DBSCAN algorithm is proposed for anomaly detection in time-series data with seasonality. For experimental evaluation, a monthly temperature dataset was employed and the analysis set forth the advantages of the modified DBSCAN over the standard DBSCAN algorithm for the seasonal datasets. From the result analysis, we may conclude that DBSCAN is used for finding the anomalies in a dataset but fails to find local anomalies in seasonal data. The proposed ...

Precision agriculture is a new approach to farming in which environmental characteristics at a sub-field level are used to guide crop production decisions. Instead of applying management actions and production inputs uniformly across entire fields, they are varied to match site-specific needs. A first step in this process is to define spatial regions having similar characteristics and to build local regression models describing the relationship between field characteristics and yield. From these yield prediction models, one can then determine optimum production input levels. Discovery of "similar" regions in fields is done by applying the DBSCAN clustering algorithm on data from more than one field ignoring spatial attributes (x and y coordinates) and the corresponding yield values. Using these models, constructed on training field regions of obtained clusters, we aim to achieve better prediction on identified regions than using global prediction models. The experimental results on real life agriculture data show observable improvements in prediction accuracy, although there are many unresolved issues in applying the proposed method in practice.

The term internet of things is a buzz word these days and as per Google survey conducted recently, it has even dominated the buzz word big data predominantly. However, IoT area is still not matured and is throwing light on lot of research issues towards the data mining researchers. Security in IoT throws several challenges because of limited resources. In this context, IoT gains importance once again from data miners towards anomaly mining or intrusion detection. Intrusion detection is classified as NPclass in the literature even today. Algorithms addressing privacy and security issues in IoT must consider the complexities involved and hence require re-attention from all researchers. One more problem faced when judging for intrusion is the use of high dimensionality, classifier choice, and distance measure. For example, the traditional distance measure, such as Euclidean misjudges the similarity. In this paper, the objective is to design a fuzzy membership function to address both dimensionality and anomaly mining so as reduce the computational complexity and increase computational accuracies of classifier algorithms. We validate the proposed measure using several experimentations on NSL-KDD and Darpa datasets using KNN, J48 and CANN using Gaussian measure. Improved accuracies of classifiers on U2R and R2L attacks have been recorded in the experimental results obtained for experiments conducted.

Traffic modeling and classification find importance in many areas such as bandwidth management, traffic analysis, prediction and engineering, network planning, Quality of Service provisioning and anomalous traffic detection. Much of research work has been done in the area of network traffic classification by application type and several classifiers are suggested. In past network traffic classification using traditional techniques such as well known port number based and payload analysis based techniques are no more effective because various applications uses port hopping and encryption technique to avoid detection. Recently machine learning techniques such as supervised, unsupervised and semi supervised techniques are used to overcome the problems of traditional techniques. In this work we use semi supervised machine learning approach to classify the network traffic using DBSCAN algorithm. This techniques uses only flow statistics to classify the network traffic. This methodology is...

Anomaly detection is a concept widely applied to numerous domains. Several techniques of anomaly detection have been developed over the years, in practice as well as research. The application of this concept has extended to diverse areas, from network intrusion detection to novelty detection in robot behavior. In the business world, the application of these techniques to fraud detection is of a special interest, driven by the great losses companies endure because of such fraudulent activities. This paper describes classification-based and clustering-based anomaly detection techniques and their applications, more specifically the application to the problem of certain fraudulent activities. As an illustration, the paper applies K-Means, a clustering-based algorithm, to a refund transactions dataset from a telecommunication company, with the intent of identifying fraudulent refunds.

The term Data Mining is used to refer the process of analyzing large datasets and then extracting the knowledge from the data. In today's world data mining has become very essential in almost every area such as market segmentation, fraud detection, market based analysis, predictions of trends and behavior, discovery of previously unknown patterns etc. These can be achieved by designing and using various data mining software. This article introduces data preprocessing, classification and clustering in association with WEKA (Waikato Environment for Knowledge Analysis) data mining tool. The steps required to use WEKA tool for different data mining tasks through various algorithms are discussed. It also summarizes the applications of different data mining tasks.

The task of network administrators to identify and determine the type of traffic traversing through the network is very critical to the rapid growth of new traffic each day. As the requirements of networks change over time, the situation of the network not able to meet some requirements is likely to occur. In a wide area network with a limited resource such as the low speed of links, frequent fragmentation of packets leading to extreme packet loss and costs is prominent resulting in the poor quality of service. As a result quantified amount of traffic flows can be classified at a time with limited features lowering the effectiveness of traffic classification. To improve upon the classification in such scenarios, we propose a hybrid semi-supervised clustering that is able to classify packet flows with restricted features and a small amount of packets while maintaining high accuracy in classification. We implement the above scenario in simulation and classify the limited flows obtaine...

The last decade has seen an explosive growth in the generation and collection of data. In the field of data mining there are various techniques are used to extract useful information from the data set. There is various estimation techniques are used in clustering methods out of these Euclidean distance and density is used for estimation. Out of these estimation techniques one another technique mass is also used. Mass based clustering gives better performance when apply with PCA technique in multidimensional and high dimensional data set. On the basis of run time behaviour the DEMassDBSCAN algorithm is better than DBSCAN clustering algorithm is discussed.

Abstract: Accurate identification and classification of network traffic according to the application that generated them is at the basis of any modern network management platform. Nowadays, many P2P applications using dynamic port numbers, masquerading techniques and encryption to avoid detection. Therefore, simple port-based and systematic analyses of packet payloads methods are rapidly inefficient. An alternative approach is to classify traffic rely on the fact that different applications have distinct behavior patterns when they communicate on a network. We present this latter approach to effectively identify groups of traffic that are similar using only transport layer statistical information. In this study, we propose a traffic monitoring scheme based on IPFIX standard that employs the clustering algorithms as a classification tool to classify network traffics using only transport layer's information. We believe that in order to build an accurate classifier, a good classif...

Traffic modeling and classification find importance in many areas such as bandwidth management, traffic analysis, prediction and engineering, network planning, Quality of Service provisioning and anomalous traffic detection. Much of research work has been done in the area of network traffic classification by application type and several classifiers are suggested. In past network traffic classification using traditional techniques such as well known port number based and payload analysis based techniques are no more effective because various applications uses port hopping and encryption technique to avoid detection. Recently machine learning techniques such as supervised, unsupervised and semi supervised techniques are used to overcome the problems of traditional techniques. In this work we use semi supervised machine learning approach to classify the network traffic using DBSCAN algorithm. This techniques uses only flow statistics to classify the network traffic. This methodology is...

This paper proposes a real-time and self-taught anomaly detection scheme for optical networks using hybrid unsupervised/supervised learning. Evaluations with an experimental dataset demonstrate that the proposed scheme can successfully identify 100% of the anomalies without any prior knowledge of abnormal network behaviors while restricting the false positive rate to be only 6.5%.

Emergence of modern techniques for scientific data collection has resulted in large scale accumulation of data pertaining to diverse fields. Generally, data mining (sometimes called data or knowledge discovery) is the process of analyzing data from different perspectives and summarizing it into useful information. Data mining software is one of a number of analytical tools for analyzing data. It allows users to analyze data from many different dimensions or angles, categorized it, and summarize the relationships identified. The development of datamining applications such as clustering has shown the need for machine learning algorithms to be applied to large scale data. In this paper we present the comparison of different clustering techniques using Waikato Environment for Knowledge Analysis or in short, WEKA. The algorithm or methods tested are Density Based Clustering, EM & K-MEANS clustering algorithms.

Network Traffic Classification is an important process in various network management activities like network planning, designing, workload characterization etc. Network traffic classification using traditional techniques such as well known port number based and payload analysis based techniques are no more effective because various applications uses port hopping and encryption technique to avoid detection. Recently machine learning techniques such as supervised, unsupervised and semi supervised techniques are used to overcome the problems of traditional techniques. In this work we use semi supervised machine learning approach and proposed distance based semi supervised clustering and probabilistic assignment technique for network traffic classification. This technique used only flow statistics to classify network traffic. It permits to build the classifier using both labeled and unlabeled instances in training dataset.

Text classification method that uses efficient similarity measures to achieve better performance is being proposed in this paper. Semi-supervised clustering is used as a complementary step to text classification and is used to identify the components in text collection. Clustering makes use of labeled texts to capture silhouettes of text clusters and unlabeled texts to adapt its centroids. The category of each text cluster is labeled by the label of texts in it. Thus here the text clustering is used to generate the classification model for the next text classification step. When a new unlabeled text is incoming, measure its similarity with the centroids of the text clusters and give its label with that of the nearest text cluster. The similarity is calculated using different similarity measures. Results and evaluations are summarized and it is found that the system provides better accuracy when a Similarity Measure for Text Processing (SMTP) used for the distance calculation.

Mobile telecommunication sector has been accelerated with GSM 1800 licenses in the Turkey. Since then, churn management has won vital importance for the GSM operators. Customers should have segmented according to their profitability for the churn management. If we know the profitable customer segments, we have chance to keep in hand the most important customers via the suitable promotions and campaigns. In this study, we implemented clustering algorithms to 250 subscribers' 100MBs call detail records(CDRs), demographic data and billing information. Density-Based Spatial Clustering of Applications with Noise (DBSCAN) compared with K-Means, Expectation Maximization and Farthest-First clustering techniques. As a result, DBSCAN has good distinct clusters for profiling customer segments.

Today maritime transportation represents 90% of international trade volume and there are more than 50,000 vessels sailing the ocean every day. Therefore, reducing maritime transportation security risks by systematically modelling and surveillance should be of high priority in the maritime domain. By statistics, majority of maritime accidents are caused by human error due to fatigue or misjudgment. Auto-vessels equipped with autonomous and semi-autonomous systems can reduce the reliance on human's intervention, thus make maritime navigation safer. This paper presents a clustering method for route planning and trajectory anomalies detection, which are the essential part of auto-vessel system design and development. In this paper, we present the development of an enhanced densitybased spatial clustering (DBSCAN) method that can be applied on historical or real-time Automatic Identification System (AIS) data, so that vessel routes can be modelled, and the trajectories' anomalies can be detected. The proposed methodology is based on developing an optimized trajectory clustering approach in two stages. Firstly, to increase the attribute dimension of the vessel's positioning data, therefore other characteristics such as velocity and direction are considered in the clustering process along with geospatial information. Secondly, the DBSCAN clustering model has been enhanced by introducing the Mahalanobis Distance metric considering the correlations of the position cluster points aiming to make the identification process more accurate as well as reducing the computational cost.

The main aim of this paper focuses on the K-means and the DBSCAN clustering techniques to identify customer behaviors and group the customers in order to improve the customer service and to remove the outliers that are existing in the dataset. We use data mining techniques for customer profiling and segmentation. Clustering can be used for identifying and grouping similar customers. Finally, our focus is to find a way to divide the objects into clusters so that we can find frequent item in each cluster to improve sales by contacting customers through message, mail or phone based on the clusters. Keywords— Clustering, K-means clustering, DBSCAN clustering, Profiling, Segmentation.

The fundamental objective behind any network intrusion detection system is to automate the detection process whenever intrusions occur in the network. The problem of the network anomaly detection is to determine, if the network incoming traffic is legitimate (or) anomalous. Automated detection systems designed to identify incoming anomalous traffic patterns usually apply widely used machine learning techniques. However, irrespective of any system model which is developed to identify anomalous traffic, all these models requires comparing anomalous and normal traffic patterns. Such comparisons implicitly depend on the ability of the underlying machine learning model to gauge the similarity between a known legitimate observation and the target. The efficiency of any network anomalous detection system depends on the use of distance (or) similarity measures and how they are actually applied. A novel distance function which can be applied to determine the similarity between two conditional feature pattern vectors is an important contribution of present research. Feature dimensionality is another important issue for any machine learning algorithm. In the present work, feature reduction is achieved using the proposed feature transformation technique. However, our approach for feature transformation uses the proposed gaussian distance function to achieve dimensionality reduction to represent the original input dataset in the new transformation space. We have also proposed new computation expressions for determining equivalent deviation and threshold in gaussian space. Experiments are performed on KDD and NSL-KDD datasets by considering widely applied classifier algorithms in various state-of-art research contributions. For performance validation of machine learning models, k-fold cross validation is applied by setting k to 10 through considering evaluation parameters such as accuracy, precision and recall. Experiment results have proved that our approach for anomaly detection that applies the proposed feature transformation technique proved comparatively better to detection methods CANN, GARUDA, and UTTAMA addressed in the recent research literature.

Information overload has raggedly increased as a result of the advances in the aspect of storage capabilities and data collection in previous years. The growth seen in the number of observation has partly cause a collapse in analytical method but the increases in the number of variable associated with each observation has grossly collapse it. The number of variables that are measured on each observation.is referred to as the dimension of the data, and a major problem of dataset containing high dimensions is that, there exist only few "important" measured variables for understanding the fundamental occurrences of interest. Hence, dimension reduction of the original data prior to any modeling of the data is of great necessity today. In this paper, a précis of K-Means, Expectation Maximization and J48 decision tree classifier is presented with a framework on the performance measurement of base classifiers with and without feature reduction. A performance evaluation was carried out based on F-Measure, Precision, Recall, True Positive Rate, False Positive Rate, ROC Area and Time taken to build model. The experiment revealed that the reduced dataset yielded improved results than the full dataset after performing classification via clustering.

Mobile telecommunication sector has been accelerated with GSM 1800 licenses in the Turkey. Since then, churn management has won vital importance for the GSM operators. Customers should have segmented according to their profitability for the churn management. If we know the profitable customer segments, we have chance to keep in hand the most important customers via the suitable promotions and campaigns. In this study, we implemented clustering algorithms to 250 subscribers' 100MBs call detail records(CDRs), demographic data and billing information. Density-Based Spatial Clustering of Applications with Noise (DBSCAN) compared with K-Means, Expectation Maximization and Farthest-First clustering techniques. As a result, DBSCAN has good distinct clusters for profiling customer segments.

Mobile telecommunication sector has been accelerated with GSM 1800 licenses in the Turkey. Since then, churn management has won vital importance for the GSM operators. Customers should have segmented according to their profitability for the churn management. If we know the profitable customer segments, we have chance to keep in hand the most important customers via the suitable promotions and campaigns. In this study, we implemented clustering algorithms to 250 subscribers' 100MBs call detail records(CDRs), demographic data and billing information. Density-Based Spatial Clustering of Applications with Noise (DBSCAN) compared with K-Means, Expectation Maximization and Farthest-First clustering techniques. As a result, DBSCAN has good distinct clusters for profiling customer segments.

Because of the increased popularity and fast expansion of the Internet as well as Internet of things, networks are growing rapidly in every corner of the society. As a result, huge amount of data is travelling across the computer networks that lead to the vulnerability of data integrity, confidentiality and reliability. So, network security is a burning issue to keep the integrity of systems and data. The traditional security guards such as firewalls with access control lists are not anymore enough to secure systems. To address the drawbacks of traditional Intrusion Detection Systems (IDSs), artificial intelligence and machine learning based models open up new opportunity to classify abnormal traffic as anomaly with a self-learning capability. Many supervised learning models have been adopted to detect anomaly from networks traffic. In quest to select a good learning model in terms of precision, recall, area under receiver operating curve, accuracy, F-score and model built time, this paper illustrates the performance comparison between Naïve Bayes, Multilayer Perceptron, J48, Naïve Bayes Tree, and Random Forest classification models. These models are trained and tested on three subsets of features derived from the original benchmark network intrusion detection dataset, NSL-KDD. The three subsets are derived by applying different attributes evaluator's algorithms. The simulation is carried out by using the WEKA data mining tool.

To detect attacks from IOT[1] and big data application using data mining techniques. Now-a-days internet can be access from anywhere using small devices such as smart phones, sensors [4] and other wearable devices etc. Always these devices will sense data such as human body[7] temperature or environment temperature or traffic data at road side etc and send this sense data to centralized server for aggregation (storage). Later this data will be used for analysis purpose such as to detect patient condition from sense patient data or to identify traffic[6] congested area. Humans will be benefitted by using above sensors and internet technologies but these will aggregate lots of data and will be called as big data and normal technique will not process such huge data and other problem is some malicious users will corrupt sensor data by attacking network or injecting extra data inside sensor sense data packet. To overcome from this problem[8], within document we are introduced a technique called CLAPP. In this technique big data attributes will be reduce by applying Dimensionality Reduction Technique. This technique will take entire data and check each column (attribute) similarity with other column and generate cluster based on similarity. If two column values are similar and related to given class then it will clustered and if not similar then that attribute will be remove out to reduce big dataset size.

With the advancement of technology and communication system, use of internet is giving at a tremendous role. This causes an exponential growth of data and traffic over the internet. So to correctly classify this traffic is a hot research area. Internet traffic classification is a very popular tool against the information detection system. Although so many methods had been develop to efficiently classify internet traffic but among them machine learning techniques are most popular. A brief survey on various supervised and unsupervised machine learning techniques applied by various researchers to solve internet traffic classification has been discussed. This paper also present various issues related to machine learning techniques that may help interested researchers to work future in this direction.

Data mining is the method of extracting the data from large database. Various data mining techniques are clustering, classification, association analysis, regression, summarization, time series analysis and sequence analysis, etc. Clustering is one of the important tasks in mining and is said to be unsupervised classification. Clustering is the techniques which is used to group similar objects or processes. In this work four clustering algorithms namely K-Means, Farthest first, EM, and Hierarchical are analyzedby the performance factors clustering accuracy, number of outliers detected and execution time. This performance analysis is carried out in BUPA (liver disorder) dataset. This work is performed in WEKA data mining tool.

Clustering is a data mining problem of dividing documents into groups, such that documents in one group are more similar than those in other groups. The aim of this study is to propose a framework for comparing the accuracy of clustering algorithms. The study applies qualitative research through document analysis to review previous clustering algorithms' comparisons so as to obtain the issues/problems with such previous comparisons. We then deduce appropriate comparisons framework that addresses the problems. The study obtained the following comparison issues: Nature of comparison, nature of data, size of data, source of data, evaluation metrics, and parameter settings. Consequently, the study proposed the rules, formulae, and procedures needed to be used in a comparison. It is recommended that applying this framework will ensure that such evaluations and comparisons are done using formal procedures that will yield dependable results. This study suggests a further study to be done to apply this framework and do a comprehensive comparison of some clustering algorithms.

In recent years, customer segmentation has become one of the most significant and useful tools for e-commerce. It plays a vital role in online product recommendation system and also helps to understand local and global wholesale or retail market. Customer segmentation refers to grouping customers into different categories based on shared characteristics such as age, location, spending habit and so on. Similarly, clustering means putting things together in such a way that similar type of things remain in the same group. Due to having similarities between these two terms, it is possible to apply clustering algorithms for ensuring satisfactory and automatic customer segmentation. Among different types of clustering algorithms, centroid based and density based are the most popular. This paper illustrates the idea of applying density based algorithms for customer segmentation beside using centroid based algorithms like kmeans. Applying DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm as one of the density based algorithms results in a meaningful customer segmentation.

In recent years, customer segmentation has become one of the most significant and useful tools for e-commerce. It plays a vital role in online product recommendation system and also helps to understand local and global wholesale or retail market. Customer segmentation refers to grouping customers into different categories based on shared characteristics such as age, location, spending habit and so on. Similarly, clustering means putting things together in such a way that similar type of things remain in the same group. Due to having similarities between these two terms, it is possible to apply clustering algorithms for ensuring satisfactory and automatic customer segmentation. Among different types of clustering algorithms, centroid based and density based are the most popular. This paper illustrates the idea of applying density based algorithms for customer segmentation beside using centroid based algorithms like kmeans. Applying DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm as one of the density based algorithms results in a meaningful customer segmentation.