Evolving Ensemble Fuzzy Classifier

Lecture Notes in Computer Science, 2014

a novel evolving fuzzy rule-based classifier, namely parsimonious classifier (pClass), is proposed in this paper. pClass can set off its learning process either from scratch with an empty rule base or from an initially trained fuzzy model. Importantly, pClass not only adopts the open structure concept, where an automatic knowledge building process can be cultivated during the training process, which is well-known as a main pillar to learn from streaming examples, but also incorporates the so-called plug-and-play principle, where all learning modules are coupled in the training process, in order to diminish the requirement of pre-or post-processing steps, undermining the firm logic of the online classifier. In what follows, pClass is equipped with the rule growing, pruning, recall and input weighting techniques, which are fully performed on the fly in the training process. The viability of pClass has been tested exploiting real-world and synthetic data streams containing some sorts of concept drifts, and compared with state-of-the-art classifiers, where pClass can deliver the most encouraging numerical results in terms of the classification rate, number of fuzzy rule, number of rule base parameters and the runtime. 1 2 2 22 21 2 1 P J J J J = . More specifically, the input weight of a particular input attribute is elicited, when this input attributes is masked. This adds up to conceive the discrimination power of each input attribute, where the input weights are assigned as follow:

Knowledge and Information Systems, 2011

Mining data streams has become an important and challenging task for a wide range of applications. In these scenarios, data tend to arrive in multiple, rapid and time-varying streams, thus constraining data mining algorithms to look at data only once. Maintaining an accurate model, e.g. a classifier, while the stream goes by requires a smart way of keeping track of the data already passed away. Such a synthetic structure has to serve two purposes: distilling the most of information out of past data and allowing a fast reaction to concept drifting, i.e. to the change of the data trend that necessarily affects the model. The paper outlines novel data structures and algorithms to tackle the above problem, when the model mined out of the data is a classifier. The introduced model and the overall ensemble architecture are presented in details, even considering how the approach can be extended for treating numerical attributes. A large part of the paper discusses the experiments and the comparisons with several existing systems. The comparisons show that the performance of our system in general, and in particular with respect to the reaction to concept drifting, is at the top level. Keywords Mining data streams • Classification • Concept drifting • Ensemble classifier • Data aggregation Definition 2.3 (Training set distribution) Given a feature vector x and a class label y, the distribution of a data stream (x i , y i) can be defined as P(x, y) = P(x) P(y|x) (1) where P(x) is the probability of generating feature vector x, while P(y|x) is the probability to assign class value y to x. As stated by Gao et al. in [30], and with reference to Eq. 1, a data stream distribution evolves from time to time in 4 different ways: No change. There is no change in the data. The model continues to be updated. Feature change. Probability P(x) changes. Some previously infrequent feature vectors become more frequent, or vice versa. The expected error depends on the specific combination between P(x) and P(y|x). The error can be higher, lower or the same. A partial model reconstruction or a retraining using new P(x) can improve accuracy. Conditional change. The conditional probability P(y|x) changes. Some feature vectors previously classified as y are now classified differently. It is likely a completely new model extraction. Dual change. Both P(x) and P(y|x) change. Since P(y|x) has evolved, it is necessary to train a completely new model. Generally, without any assumption about data distribution, in none of the aforementioned four cases, the expected error can be used as an indication of concept drifting. Therefore, it is very difficult to find a general correlation between the expected error of the previously extracted model and any type of concept drifting. Even though the expected error of a model does not change, a variation in the distribution can occur and the classifier is not updated to the current data. 2.2 Related work Data streams processing has recently become an important research domain. This aspect is testified by the number of papers currently published and by the real applications readily

Pattern Recognition, 2021

Ensemble learning has been widely applied to both batch data classification and streaming data classification. For the latter setting, most existing ensemble systems are homogenous, which means they are generated from only one type of learning model. In contrast, by combining several types of different learning models, a heterogeneous ensemble system can achieve greater diversity among its members, which helps to improve its performance. Although heterogeneous ensemble systems have achieved many successes in the batch classification setting, it is not trivial to extend them directly to the data stream setting. In this study, we propose a novel HEterogeneous Ensemble Selection (HEES) method, which dynamically selects an appropriate subset of base classifiers to predict data under the stream setting. We are inspired by the observation that a well-chosen subset of good base classifiers may outperform the whole ensemble system. Here, we define a good candidate as one that expresses not only high predictive performance but also high confidence in its prediction. Our selection process is thus divided into two sub-processes: accurate-candidate selection and confident-candidate selection. We define an accurate candidate in the stream context as a base classifier with high accuracy over the current concept, while a confident candidate as one with a confidence score higher than a certain threshold. In the first sub-process, we employ the prequential accuracy to estimate the performance of a base classifier at a specific time, while in the latter sub-process, we propose a new measure to quantify the predictive confidence and provide a method to learn the threshold incrementally. The final ensemble is formed by taking the intersection of the sets of confident classifiers and accurate classifiers. Experiments on a wide range of data streams show that the proposed method achieves competitive performance with lower running time in comparison to the state-of-the-art online ensemble methods.

The Scientific World Journal, 2015

The treatment of large data streams in the presence of concept drifts is one of the main challenges in the field of data mining, particularly when the algorithms have to deal with concepts that disappear and then reappear. This paper presents a new algorithm, called Fast Adapting Ensemble (FAE), which adapts very quickly to both abrupt and gradual concept drifts, and has been specifically designed to deal with recurring concepts. FAE processes the learning examples in blocks of the same size, but it does not have to wait for the batch to be complete in order to adapt its base classification mechanism. FAE incorporates a drift detector to improve the handling of abrupt concept drifts and stores a set of inactive classifiers that represent old concepts, which are activated very quickly when these concepts reappear. We compare our new algorithm with various well-known learning algorithms, taking into account, common benchmark datasets. The experiments show promising results from the pr...

2013 IEEE Second International Conference on Image Information Processing (ICIIP-2013), 2013

Data Streams are data instances which arrive at a very rapid rate with varying concepts. Many online ensembles of classifiers were developed which handled the drifting concepts and were proved to be better than a single classifier system. In our work, we will discuss our new approach, Early Dynamic Weighted Majority and will empirically prove it to be better than the existing online ensemble approaches. Empirical results would prove that all these online approaches can be quite competitive, and show good accuracy and speed in handling and identifying drifts in data.

Procedia Computer Science, 2014

The problem addressed in this paper concerns mining data streams with concept drift. The goal of the paper is to propose and validate a new approach to mining data streams with concept-drift using the ensemble classifier constructed from the one-class base classifiers. It is assumed that base classifiers of the proposed ensemble are induced from incoming chunks of the data stream. Each chunk consists of prototypes and can be updated using instance selection technique when a new data have arrived. When a new data chunk is formed, ensemble model is also updated on the basis of weights assigned to each one-class classifier. The proposed approach is validated experimentally.

International Journal of Electrical and Computer Engineering (IJECE), 2016

Incremental Learning on non stationary distribution has been shown to be a very challenging problem in machine learning and data mining, because the joint probability distribution between the data and classes changes over time. Many real time problems suffer concept drift as they changes with time. For example, an advertisement recommendation system, in which customer's behavior may change depending on the season of the year, on the inflation and on new products made available. An extra challenge arises when the classes to be learned are not represented equally in the training data i.e. classes are imbalanced, as most machine learning algorithms work well only when the training data is balanced. The objective of this paper is to develop an ensemble based classification algorithm for non-stationary data stream (ENSDS) with focus on two-class problems. In addition, we are presenting here an exhaustive comparison of purposed algorithms with state-of-the-art classification approaches using different evaluation measures like recall, fmeasure and g-mean.

Classification in the data stream is the challenging fact for the data mining community. In this paper, we tackle four major challenges which are infinite length, concept drift, concept evolution, and feature evolution. As we know that the data streams are huge in amount, so practically it is not possible to store the data and used it for the training purpose. The results of changes in the underlying concepts are occurred because of concept drift, which is the general observable fact in the data streams. The result of new classes surfacing in the data streams occurs because of concept evaluation. The feature evaluation generally occurs in many streams like text streams, in text streams new features emerge as stream advancement. Many existing methods of the data stream classification tackle only first two challenges and ignore last two challenges. Here in this paper we proposed an ensemble classification skeleton, in which each classifier is prepared with a novel class detector to tackle the concept drift and concept evolution. We also proposed the feature set homogenization methods for feature evaluation. We improve the component of novel class detection by making it more adaptive to the evolving stream, and enable it to notice more than one novel class at a time. As comparing with the existing methods of the novel class detector method the efficiency of the proposed method is more than the existing one.

IEEE Access

In a data stream environment, classification models must effectively and efficiently handle concept drift. Ensemble methods are widely used for this purpose; however, the ones available in the literature either use a large data chunk to update the model or learn the data one by one. In the former, the model may miss the changes in the data distribution, while in the latter, the model may suffer from inefficiency and instability. To address these issues, we introduce a novel ensemble approach based on the Broad Learning System (BLS), where mini chunks are used at each update. BLS is an effective lightweight neural architecture recently developed for incremental learning. Although it is fast, it requires huge data chunks for effective updates and is unable to handle dynamic changes observed in data streams. Our proposed approach, named Broad Ensemble Learning System (BELS), uses a novel updating method that significantly improves best-inclass model accuracy. It employs an ensemble of output layers to address the limitations of BLS and handle drifts. Our model tracks the changes in the accuracy of the ensemble components and reacts to these changes. We present our mathematical derivation of BELS, perform comprehensive experiments with 35 datasets that demonstrate the adaptability of our model to various drift types, and provide its hyperparameter, ablation, and imbalanced dataset performance analysis. The experimental results show that the proposed approach outperforms 10 state-of-the-art baselines, and supplies an overall improvement of 18.59% in terms of average prequential accuracy. INDEX TERMS Data stream mining, concept drift, ensemble learning, neural networks, big data.

International Journal of Engineering Sciences and Research Technology, 2016

Today, rapid growth in hardware technology has provided a means to generate huge volume of data continuously. In most of the real time data stream application data usually reach very rapidly that flows continuously in real time environment .This incoming data streams comprises of several important and interesting patterns underneath. However, mining an essential data out from this data stream, has some major challenges such as infinite length, concept evolution and concept-drift .Earlier studies, carried till now, have been mainly focusing on building accurate classification model. But, keeping glance over the huge amount of classifiers while prediction, require more response time, which makes ensemble approach being impractical for many real-world times critical data stream applications. In order to over this problem, we propose a new enhanced indexing structure that organizes all classifiers of ensemble in order to get fast prediction response in lesser amount of time .In addition to this, ensemble model is updated continuously by integrating new classifiers, while adapting to new trends. Experimental results and theoretical analysis on both real-worlds as well as synthetic data streams exhibit the better performance of our method over the existing techniques.