Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Motivation 1.DATA-STREAM Classification
Example Applications of Data-Stream Classification
Standard Data Classification
Data-Stream Mining
Data-Stream Classification
Decision Trees for Data Streams
Concept-Drift Detection in Data Streams
Limitations of Stream-Classification Algorithms
Number of Training Data Records
Sampling for Labelled Data
Datasets
Implementation and Experimental Settings
Synthetic Data
Data Sets
Real Data
Results and Discussion
Summary
Conclusions
Future Work
References
All Topics
Computer Science
Machine Learning

Streaming Random Forests

Profile image of hanady abdulsalamhanady abdulsalam

2007, 11th International Database Engineering and Applications Symposium (IDEAS 2007)

https://doi.org/10.1109/IDEAS.2007.4318108
visibility

…

description

146 pages

link

1 file

Abstract

Recent research addresses the problem of data-stream mining to deal with applications that require processing huge amounts of data such as sensor data analysis and financial applications. Data-stream mining algorithms incorporate special provisions to meet the requirements of stream-management systems, that is stream algorithms must be online and incremental, processing each data record only once (or few times); adaptive to distribution changes; and fast enough to accommodate high arrival rates. We consider the problem of data-stream classification, introducing an online and incremental stream-classification ensemble algorithm, Streaming Random Forests, an extension of the Random Forests algorithm by Breiman, which is a standard classification algorithm. Our algorithm is designed to handle multi-class classification problems. It is able to deal with data streams having an evolving nature and a random arrival rate of training/test data records. The algorithm, in addition, automatically adjusts its parameters based on the data seen so far. Experimental results on real and synthetic data demonstrate that the algorithm gives a successful behavior. Without losing classification accuracy, our algorithm is able to handle multi-class problems for which the underlying class boundaries drift, and handle the case when blocks of training records are not big enough to build/update the classification model.

Related papers

Critical evaluation of classifiers in data stream mining
Dattatraya Adane

International Journal of Engineering & Technology

Over past decade there has been a significant increase in the volume of online data. Extracting meaningful knowledge from this high volume data is considered as important aspect of research. It is very difficult to completely store full data, because of its perpetual nature. Therefore, analysis is needed while the “data is moving”. This moving data is known as data stream and analyzing it without storing it completely is termed as data stream mining. In recent years, many new techniques have been proposed to overcome the challenges of data stream mining. In this paper, we review the operation of popular streaming algorithms highlighting their strength and weaknesses. We also evaluate the classifiers used in these algorithms against two popular benchmark datasets namely (a) forest cover (forest) and (b) german credit available at UCI repository. Finally, we present our critical observation and draw conclusions on the basis of our analysis.

View PDFchevron_right
New ensemble methods for evolving data streams
Albert Bifet

2009

Advanced analysis of data streams is quickly becoming a key area of data mining research as the number of applications demanding such processing increases. Online mining when such data streams evolve over time, that is when concepts drift or change completely, is becoming one of the core issues. When tackling non-stationary concepts, ensembles of classifiers have several advantages over single classifier methods: they are easy to scale and parallelize, they can adapt to change quickly by pruning under-performing parts of the ensemble, and they therefore usually also generate more accurate concept descriptions. This paper proposes a new experimental data stream framework for studying concept drift, and two new variants of Bagging: ADWIN Bagging and Adaptive-Size Hoeffding Tree (ASHT) Bagging. Using the new experimental framework, an evaluation study on synthetic and real-world datasets comprising up to ten million examples shows that the new ensemble methods perform very well compared to several known methods.

View PDFchevron_right
Incremental Learning Algorithm for Dynamic Data Streams
DR. A. GOVARDHAN

2008

The recent advances in hardware and software have enabled the capture of different measurements of data in a wide range of fields. These measurements are generated continuously and in a very high fluctuating data rates. Examples include sensor networks, web logs, and computer network traffic. The storage, querying and mining of such data sets are highly computationally challenging tasks. Mining data streams is concerned with extracting knowledge structures represented in models and patterns in non stopping streams of information. The research in data stream mining has gained a high attraction due to the importance of its applications and the increasing generation of streaming information. Applications of data stream analysis can vary from critical scientific and astronomical applications to important business and financial ones. Algorithms, systems and frameworks that address streaming challenges have been developed from the past few years. This paper presents a system for induction of forest of functional trees from data streams able to detect concept drift. The Ultra Fast Forest of Trees (UFFT)is an incremental algorithm, which works online, processing each example in constant time, and performing a single scan over the training examples. It uses analytical techniques to choose the splitting criteria, and the information gain to estimate the merit of each possible splitting-test. For multi-class problems the algorithm builds a binary tree for each possible pair of classes, leading to a forest of trees. Decision nodes and leaves contain naive-Bayes classifiers playing different roles during the induction process. Naive-Bayes in leaves are used to classify test examples. Naive-Bayes in inner nodes play two different roles. They can be used as multivariate splitting-tests if chosen by the splitting criteria, and used to detect changes in the classdistribution of the examples that traverse the node. When a change in the class-distribution is detected,all the subtree rooted at that node will be pruned. The use of naive-Bayes classifiers at leaves to classify test examples, the use of splitting-tests based on the outcome of naive-Bayes, and the use of naive-Bayes classifiers at decision nodes to detect changes in the distribution of the examples are directly obtained from the sufficient statistics required to compute the splitting criteria, without no additional computations. This aspect is a main advantage in the context of high-speed data streams. This methodology was tested with artificial and real-world data sets. The experimental results show a very good performance in comparison to a batch decision tree learner, and high capacity to detect drift in the distribution of the examples.

View PDFchevron_right
Classification of Evolving Stream Data Using Improved Ensemble Classifier
Seema Shedole

International Journal of Data Mining & Knowledge Management Process, 2012

Data mining is a user-centric process that is used to extract useful patterns from large volumes of data. With the growth of the Internet, a data stream is today a key area of advanced analysis and data mining. Handling data streams is a difficult task due to the variations in the data and the frequent occurrences of concept drifts. No single classifier can be relied upon to correctly classify data stream data since they are developed through a specific learning approach. Hence we use a multi-chunk ensemble of classifiers to classify evolving data streams and improve the prediction accuracy over single classifiers. We evaluate our ensemble on synthetic as well as real time data, compute the precision and represent it graphically using both majority voting as well as new proposed weighted averaging and compare its performance against individual classifiers. Current techniques include a single chunk approach, where the entire set of data is considered as a whole, and the used method shows better efficiency.

View PDFchevron_right
Adapted One-vs-All Decision Trees for Data Stream Classification
Sattar Hashemi

2009

One-vs-all (OVA) classifiers learn k individual binary classifiers, each one to distinguish the instances of a single class from the instances of all other classes. To classify a new instance, the k classifiers are run and the one that returns the highest confidence is chosen. Thus OVA is different from existing data stream classification schemes whose majority use multiclass classifiers, each one to discriminate among all the classes. This paper advocates some outstanding advantages of OVA for data stream classification. First, there is low error correlation and hence high diversity among OVA's component classifiers, which leads to high classification accuracy. Second, OVA is adept at accommodating new class labels that often appear in data streams. However, there also remain many challenges to deploy traditional OVA for classifying data streams. First, traditional OVA does not handle concept change, a key feature of data streams. Second, as every instance is fed to all component classifiers, OVA is known as an inefficient model. Third, OVA's classification accuracy is adversely affected by the imbalanced class distributions in data streams. This paper addresses those key challenges and consequently proposes a new OVA scheme that is adapted for data stream classification. Theoretical analysis and empirical evidence reveal that the adapted OVA can offer faster training, faster updating and higher classification accuracy than many existing popular data stream classification algorithms. We expect these results to be of interest to researchers and practitioners because they suggest a simple but very elegant and effective alternative to existing classification schemes for data streams.

View PDFchevron_right
Ensemble learning for data stream analysis: A survey
Joao Gama

Information Fusion, 2017

In many applications of information systems learning algorithms have to act in dynamic environments where data are collected in the form of transient data streams. Compared to static data mining, processing streams imposes new computational requirements for algorithms to incrementally process incoming examples while using limited memory and time. Furthermore, due to the non-stationary characteristics of streaming data, prediction models are often also required to adapt to concept drifts. Out of several new proposed stream algorithms, ensembles play an important role, in particular for nonstationary environments. This paper surveys research on ensembles for data stream classification as well as regression tasks. Besides presenting a comprehensive spectrum of ensemble approaches for data streams, we also discuss advanced learning concepts such as imbalanced data streams, novelty detection, active and semi-supervised learning, complex data representations and structured outputs. The paper concludes with a discussion of open research problems and lines of future research.

View PDFchevron_right
Scalable real-time classification of data streams with concept drift
Frederic Stahl

Future Generation Computer Systems, 2017

h i g h l i g h t s • A real-time data stream classifier adaptive to concept drift and robust to noise. • A parallel implementation of the real-time data stream classifier. • A discussion about using open source Big Data technologies for data stream mining.

View PDFchevron_right
Stream Data Mining: Platforms, Algorithms, Performance Evaluators and Research Trends
Sonali Agarwal

International Journal of Database Theory and Application, 2016

Streaming data are potentially infinite sequence of incoming data at very high speed and may evolve over the time. This causes several challenges in mining large scale high speed data streams in real time. Hence, this field has gained a lot of attention of researchers in previous years. This paper discusses various challenges associated with mining such data streams. Several available stream data mining algorithms of classification and clustering are specified along with their key features and significance. Also, the significant performance evaluation measures relevant in streaming data classification and clustering are explained and their comparative significance is discussed. The paper illustrates various streaming data computation platforms that are developed and discusses each of them chronologically along with their major capabilities. This paper clearly specifies the potential research directions open in high speed large scale data stream mining from algorithmic, evolving nature and performance evaluation measurement point of view. Finally, Massive Online Analysis (MOA) framework is used as a use case to show the result of key streaming data classification and clustering algorithms on the sample benchmark dataset and their performances are critically compared and analyzed based on the performance evaluation parameters specific to streaming data mining.

View PDFchevron_right
Learning decision trees from dynamic data streams
Joao Gama

Journal of Universal Computer Science, 2005

This paper presents a system for induction of forest of functional trees from data streams able to detect concept drift. The Ultra Fast Forest of Trees (UFFT) is an incremental algorithm, which works online, processing each example in constant time, and performing a single scan over the training examples. It uses analytical techniques to choose the splitting criteria, and the information gain to estimate the merit of each possible splitting-test. For multi-class problems the algorithm builds a binary tree for each possible pair of classes, leading to a forest of trees. Decision nodes and leaves contain naive-Bayes classifiers playing different roles during the induction process. Naive-Bayes in leaves are used to classify test examples. Naive-Bayes in inner nodes play two different roles. They can be used as multivariate splitting-tests if chosen by the splitting criteria, and used to detect changes in the class-distribution of the examples that traverse the node. When a change in the class-distribution is detected, all the sub-tree rooted at that node will be pruned. The use of naive-Bayes classifiers at leaves to classify test examples, the use of splitting-tests based on the outcome of naive-Bayes, and the use of naive-Bayes classifiers at decision nodes to detect changes in the distribution of the examples are directly obtained from the sufficient statistics required to compute the splitting criteria, without no additional computations. This aspect is a main advantage in the context of high-speed data streams. This methodology was tested with artificial and real-world data sets. The experimental results show a very good performance in comparison to a batch decision tree learner, and high capacity to detect drift in the distribution of the examples.

View PDFchevron_right
An iterative boosting-based ensemble for streaming data classification
Joao Bertini

Information Fusion, 2019

Among the many issues related to data stream applications, those involved in predictive tasks such as classification and regression, play a significant role in Machine Learning (ML). The so-called ensemble-based approaches have characteristics that can be appealing to data stream applications, such as easy updating and high flexibility. In spite of that, some of the current approaches consider unsuitable ways of updating the ensemble along with the continuous stream processing, such as growing it indefinitely or deleting all its base learners when trying to overcome a concept drift. Such inadequate actions interfere with two inherent characteristics of data streams namely, its possible infinite length and its need for prompt responses. In this paper, a new ensemblebased algorithm, suitable for classification tasks, is proposed. It relies on applying boosting to new batches of data aiming at maintaining the ensemble by adding a certain number of base learners, which is established as a function of the current ensemble accuracy rate. The updating mechanism enhances the model flexibility, allowing the ensemble to gather knowledge fast to quickly overcome high error rates, due to concept drift, while maintaining satisfactory results by slowing down the updating rate in stable concepts. Results comparing the proposed ensemble-based algorithm against eight other ensembles found in the literature show that the proposed algorithm is very competitive when dealing with data stream classification.

View PDFchevron_right

References (72)

  1. Pandora music station. http://www.pandora.com/.
  2. Random Forest FORTRAN Code. Available at http://www.stat.berkeley.edu/ breiman/RandomForests/cc_software.htm/.
  3. Forest CoverType dataset. Available at http://kdd.ics.uci.edu/.
  4. The Sixth Data Release of the Sloan Digital Sky Survey. The Astrophysical Journal Supplement Series, 175:297-313, April 2008.
  5. D. Abadi, D. Carney, U. Cetintemel, M. Cherniack, C. Convey, S. Lee, M. Stonebraker, N. Tatbul, and S. Zdonik. Aurora: A new model and architecture for data stream management. Journal of Very Large Data Bases (VLDB), 12(2):120-139, August 2003.
  6. H. Abdulsalam, D. Skillicorn, and P. Martin. Mining data-streams. In P. Poncelet, F. Masseglia, and M. Tessiere, editors, Success and New Directions in Data Mining, pages 302-324. Idea Group Inc. (IGI), October 2007.
  7. H. Abdulsalam, D. Skillicorn, and P. Martin. Streaming random forests. In Proceedings of the 11th International Database Engineering and Applications Symposium(IDEAS), pages 225-232, September 2007.
  8. H. Abdulsalam, D. Skillicorn, and P. Martin. Classifying evolving data streams using dynamic streaming random forests. In Proceedings of the 19th International Conference on Database and Expert Systems Applications (DEXA), to appear September 2008.
  9. C. Aggarwal, J. Han, J. Wang, and P. Yu. A framework for clustering evolving data streams. In Proceedings of 29th International Conference on Very Large Data Bases(VLDB), pages 81-92. Berlin, Germany, September 2003.
  10. C. Aggarwal, J. Han, J.and Wang, and P. Yu. A framework for high dimensional pro- jected clustering of data streams. In Proceedings of the 30th International Conference on Very Large Data Bases (VLDB), pages 852-863. Toronto, Canada, August 2004.
  11. A. Arasu, B. Babcock, S. Babu, M. Datar, K. Ito, R. Motwani, I. Nishizawa, U. Sri- vastava, D. Thomas, R. Varma, and J. Widom. STREAM: The Stanford Stream Data Manager. IEEE Data Engineering Bulletin, 26(1):19-26, March 2003.
  12. B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom. Models and issues in data stream systems. In Proceedings of the 21st ACM SIGACT-SIGMOD-SIGART Sym- posium on Principles of Database Systems (PODS), pages 1-16. Madison, Wisconsin, June 2002.
  13. B. Babcock, M. Datar, R. Motwani, and L. O'Callaghan. Maintaining variance and k-medians over data stream windows. In Proceedings of the 22nd ACM SIGACT- SIGMOD-SIGART Symposium on Principles of Database Systems(PODS), pages 234- 243. San Diego, CA, June 2003.
  14. D. Barbará. Requirements for clustering data streams. ACM SIGKDD Knowledge Discovery in Data and Data Mining Explorations Newsletter, 3(2):23-27, January 2002.
  15. J. Blackard. Comparison of Neural Networks and Discriminant Analysis in Predict- ing Forest Cover Types. PhD thesis, Department of Forest Sciences. Colorado State University, Fort Collins, Colorado, 1998.
  16. P. Bonnet, J. E. Gehrke, , and P. Seshadri. Towards sensor database systems. In Proceedings of the 2nd International Conference on Mobile Data Management (MDM), pages 3-14. Hong Kong, China, January 2001.
  17. L. Breiman. Bagging predictors. Technical report, Statistics Department, University of California, Berkeley, September 1994.
  18. L. Breiman. Random forests. Machine Learning, 45(1):5-32, October 2001.
  19. L. Breiman, J. Friedman, R. Olshen, and C. Stone. Classification and Regression Trees. Wadsworth International, Belmont, CA, 1984.
  20. A. Bulut and A. Singh. A unified framework for monitoring data streams in real time. In Proceedings of the 21st International Conference on Data Engineering (ICDE), pages 44-55. Tokyo, Japan, April 2005.
  21. G. C. and L. Grossman. GenIc: A single pass generalized incremental algorithm for clustering. In Proceedings of the 1st Workshop on Secure Data Management (SDM). Toronto, Canada, April 2004.
  22. Y. Chen, G. Dong, J. Han, B. W. Wah, and J. Wang. Multi-dimensional regression analysis of time-series data streams. In Proceedings of the 28th international conference on Very Large Data Bases(VLDB), pages 323-334. Hong Kong, China, August 2002.
  23. F. Chu, Y. Wang, and C. Zaniolo. An adaptive learning approach for noisy data streams. In Proceedings of the 4th IEEE International Conference on Data Mining (ICDM), pages 351-354. Brighton, UK, November 2004.
  24. T. Dasu, S. Krishnan, S. Venkatasubramanian, and K. Yi. An information-theoretic approach to detecting changes in multi-dimensional data streams. In Proceedings of the 38th Symposium on the Interface of Statistics. Pasadena, CA, May 2006.
  25. M. Datar, A. Gionis, P. Indyk, and R. Motwani. Maintaining stream statistics over sliding windows. In Proceedings of the 13th Annual ACM-SIAM Symposium on Dis- crete Algorithms (SODA), pages 635-644. San Francisco, CA, January 2002.
  26. P. Domingos and G. Hulten. Mining high-speed data streams. In Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 71-80. Boston, MA, August 2000.
  27. M. H. Dunham. Data Mining. Introductory and Advanced Topics. Prentice Hall, 2003.
  28. W. Fan. A systematic data selection to mine concept-drifting data streams. In Pro- ceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 128-137. Seattle, Washington, August 2004.
  29. W. Fan, Y. Huang, and P. S. Yu. Decision tree evolution using limited number of labeled data items from drifting data streams. In Proceedings of the 4th IEEE Interna- tional Conference on Data Mining (ICDM), pages 379-382. Brighton, UK, November 2004.
  30. Y. Freund and R. E. Schapire. Experiments with new boosting algorithms. In Pro- ceedings of the 13th International Conference on Machine Learning (ICML), pages 148-146. Bari, Italy, July 1996.
  31. M. Gaber, S. Krishnaswamy, and A. Zaslavsky. Cost-efficient mining techniques for data streams. In Proceedings of the 1st Australasian Workshop on Data Mining and Web Intelligence (DMWI), pages 81-92. Dunedin, New Zealand, January 2004.
  32. J. Gama, P. Medas, and R. Rocha. Forest trees for on-line data. In Proceedings of the 2004 ACM Symposium on Applied Computing (SAC), pages 632-636. Nicosia, Cyprus, March 2004.
  33. J. Gama, P. Medas, and P. Rodrigues. Learning decision trees from dynamic data streams. In Proceedings of the 2005 ACM symposium on Applied computing (SAC), pages 573-577. Santa Fe, New Mexico, March 2005.
  34. J. Gama, R. Rocha, and P. Medas. Accurate decision trees for mining high-speed data streams. In Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 523-528. Washington, DC, Au- gust 2003.
  35. W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of American Statistical Association, 58(1):13-30, 1963.
  36. G. Hulten, L. Spencer, and P. Domingos. Mining time-changing data streams. In Pro- ceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data mining (KDD), pages 97-106. San Francisco, CA, August 2001.
  37. L. Jia, Z. Wang, N. Lu, X. Xu, D. Zhou, and Y. Wang. Rfiminer: A regression-based algorithm for recently frequent patterns in multiple time granularity data streams. Applied Mathematics and Computation, 185(2):769-783, February 2007.
  38. R. Jin and G. Agrawal. Efficient decision tree construction on streaming data. In Proceedings of International Conference on Knowledge Discovery and Data Mining (SIGKDD), pages 571-576. Washington, DC, August 2003.
  39. M. Kantardzic. Data Mining. Concepts, Models, Methods, and Algorithms. IEEE Press, 2003.
  40. R. M. Karp, C. H. Papadimitriou, and S. Shenker. A simple algorithm for finding fre- quent elements in streams and bags. ACM Transactions on Database Systems (TODS), 28(1):51-55, March 2003.
  41. D. Kifer, S. Ben-David, and J. Gehrke. Detecting changes in data streams. In Proceed- ings of the 30th International Conference on Very Large Data Bases (VLDB), pages 180-191. Toronto, Canada, August 2004.
  42. B. Krishnamurthy, S. Sen, Y. Zhang, and Y. Chen. Sketch-based change detection: methods, evaluation, and applications. In Proceedings of the 3rd ACM SIGCOMM In- ternet Measurement Conference (IMC), pages 234-247. Miami Beach, Florida, October 2003.
  43. W. Li, X. Jin, and X. Ye. Detecting change in data stream: Using sampling technique. In Proceedings of the Third International Conference on Natural Computation (ICNC), pages 130-134. Haikou, Hainan, China, August 2007.
  44. Z. Li, T. Wang, R. Wang, Y. Yan, and H. Chen. A new fuzzy decision tree classifi- cation method for mining high-speed data streams based on binary search trees. In Proceedings of the First International Frontiters of Algorithmics WorkShop (FAW), pages 216-227. Lanzhou, China, August 2007.
  45. Y. Liu, J. Cai, J. Yin, and A. W.-C. Fu. Clustering text data streams. Journal of Computer Science Technology, 23(1):112-128, January 2008.
  46. L.Rokach. Ensemble methods for classifiers. In O. Maimon and L. Rokach, editors, The Data Mining and Knowledge Discovery Handbook, pages 957-980. Springer Science and Business Media Inc., 2005.
  47. L.Rokach and O. Maimon. Decision trees. In O. Maimon and L. Rokach, editors, The Data Mining and Knowledge Discovery Handbook, pages 165-192. Springer Science and Business Media Inc., 2005.
  48. S. McConnell and D. Skillicorn. Distributed Data Mining for Astrophysical Datasets. In P. Shopbell, M. Britton, and R. Ebert, editors, Proceedings of the Astronomical Data Analysis Software and Systems XIV, volume 347 of Astronomical Society of the Pacific Conference Series, pages 360-364, December 2005.
  49. G. Melli. (SCDS-A) Synthetic classification data set generator. Simon Fraser Univer- sity, School of Computer Science, 1997.
  50. A. Metwally, D. Agrawal, and A. El-Abbadi. Efficient computation of frequent and top-k elements in data streams. In Proceedings of the 10th International Conference on Database Theory (ICDT), pages 398-412. Edinburgh, UK, January 2005.
  51. S. Nassar and J. Sander. Effective summarization of multi-dimensional data streams for historical stream mining. In Proceedings of the 19th International Conference on Scientific and Statistical Database Management (SSDBM), pages 30-39. Banff, AB, Canada, July 2007.
  52. K. Nishida, K. Yamauchi, and T. Omori. ACE: Adaptive classifiers-ensemble system for concept-drifting environments. In Proceedings of the 6th International Workshop on Multiple Classifier Systems (MCS), pages 176-185. Seaside, CA, USA, June 2005.
  53. C. Olaru and L. Wehenkel. A complete fuzzy decision tree technique. Journal of Fuzzy Sets and Systems, 138(2):221-254, September 2003.
  54. P.Buhlmann. Bagging, boosting and ensemble methods. In J. Gentle, W. Hardle., and Y. Mori, editors, Handbook of Computational Statistics. Concepts and Methods, pages 877-907. Springer Science and Business Media Inc., 2004.
  55. C. E. Shannon. A mathematical theory of communication. Proceedings of the ACM SIGMOBILE Mobile Computing and Communications Review, 5(1):355, January 2001.
  56. W. Street and Y. Kim. A streaming ensemble algorithm (SEA) for large-scale classifica- tion. In Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 377-382. San Francisco, CA, USA, August 2001.
  57. M. Sullivan and A. Heybey. Tribeca: A system for managing large databases of network traffic. In Proceedings of the USENIX Annual Technical Conference, pages 13-24. New Orleans, Louisiana, June 1998.
  58. Y. Sun, G. Mao, X. Liu, and C. Liu. Mining concept drifts from data streams based on multi-classifiers. In Proceedings of the 21st International Conference on Advanced In- formation Networking and Applications Workshops (AINAW), pages 257-263. Niagara Falls, Canada, May 2007.
  59. W. G. Teng, M. S. Chen, and P. S. Yu. A regression-based temporal pattern mining scheme for data streams. In Proceedings of the 29th international conference on Very large data bases (VLDB), pages 93-104. Berlin, Germany, September 2003.
  60. C. J. Tsai, C. I. Lee, and W. P. Yang. An efficient and sensitive decision tree approach to mining concept-drifting data streams. Informatica, 19(1):135-156, February 2008.
  61. K. Udommanetanakit, T. Rakthanmanon, and K. Waiyamai. E-Stream: Evolution- based technique for stream clustering. In Proceedings of The Third International Con- ference on Advanced Data Mining and Applications (ADMA), pages 605-615. Harbin, China, August 2007.
  62. P. Vorburger and A. Bernstein. Entropy-based concept shift detection. In Proceed- ings of the Sixth International Conference on Data Mining (ICDM), pages 1113-1118. Hong-Kong, December 2006.
  63. H. Wang, W. Fan, P. Yu, and J. Han. Mining concept-drifting data streams using ensemble classifiers. In Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 226-235. Washington, DC, August 2003.
  64. T. Wang, Z. Li, X. Hu, Y. Yan, and H. Chen. A new decision tree classification method for mining high-speed data streams based on threaded binary search trees. In Proceedings of the 11th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), pages 256-267. Nanjing, China, May 2007.
  65. T. Wang, Z. Li, Y. Yan, H. Chen, and J. Yu. An efficient classification system based on binary search trees for data streams mining. In Proceedings of the Second International Conference on Systems (ICONS), pages 15-15. Sainte-Luce, Martinique, April 2007.
  66. X. Wang, H. Liu, and J. Han. Finding frequent items in data streams using hierarchical information. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (ISIC), pages 431-436, October 2007.
  67. I. H. Witten and E. Frank. Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, San Francisco, 2nd edition, 2005.
  68. Y. Zhang and X. Jin. An automatic construction and organization strategy for ensem- ble learning on data streams. ACM SIGMOD Record, 35(3):28-33, September 2006.
  69. A. Zhou, S. Qin, and W. Qian. Adaptively detecting aggregation bursts in data streams. In Proceedings of the 10th International Conference of Database Systems for Advanced Applications (DASFAA), pages 435-446. Beijing, China, April 2005.
  70. X. Zhu, X. Wu, and Y. Yang. Dynamic classifier selection for effective mining from noisy data streams. In Proceedings of the 4th IEEE International Conference on Data Mining (ICDM), pages 305-312. Brighton, UK, November 2004.
  71. Y. Zhu and D. Shasha. Statistical monitoring of thousands of data streams in real time. In Proceedings of the 28th International Conference on Very Large Data Bases (VLDB), pages 358-369. Hong Kong, China, August 2002.
  72. Y. Zhu and D. Shasha. Efficient elastic burst detection in data streams. In Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 336-345. Washington, DC, August 2003.

Related papers

Classifying Evolving Data Streams Using Dynamic Streaming Random Forests
hanady abdulsalam

Lecture Notes in Computer Science, 2008

We consider the problem of data-stream classification, introducing a stream-classification algorithm, Dynamic Streaming Random Forests, that is able to handle evolving data streams using an entropy-based drift-detection technique. The algorithm automatically adjusts its parameters based on the data seen so far. Experimental results show that the algorithm handles multi-class problems for which the underlying class boundaries drift, without losing accuracy.

View PDFchevron_right
Stream mining: a novel architecture for ensemble-based classification
Franco Turini

Knowledge and Information Systems, 2011

Mining data streams has become an important and challenging task for a wide range of applications. In these scenarios, data tend to arrive in multiple, rapid and time-varying streams, thus constraining data mining algorithms to look at data only once. Maintaining an accurate model, e.g. a classifier, while the stream goes by requires a smart way of keeping track of the data already passed away. Such a synthetic structure has to serve two purposes: distilling the most of information out of past data and allowing a fast reaction to concept drifting, i.e. to the change of the data trend that necessarily affects the model. The paper outlines novel data structures and algorithms to tackle the above problem, when the model mined out of the data is a classifier. The introduced model and the overall ensemble architecture are presented in details, even considering how the approach can be extended for treating numerical attributes. A large part of the paper discusses the experiments and the comparisons with several existing systems. The comparisons show that the performance of our system in general, and in particular with respect to the reaction to concept drifting, is at the top level. Keywords Mining data streams • Classification • Concept drifting • Ensemble classifier • Data aggregation Definition 2.3 (Training set distribution) Given a feature vector x and a class label y, the distribution of a data stream (x i , y i) can be defined as P(x, y) = P(x) P(y|x) (1) where P(x) is the probability of generating feature vector x, while P(y|x) is the probability to assign class value y to x. As stated by Gao et al. in [30], and with reference to Eq. 1, a data stream distribution evolves from time to time in 4 different ways: No change. There is no change in the data. The model continues to be updated. Feature change. Probability P(x) changes. Some previously infrequent feature vectors become more frequent, or vice versa. The expected error depends on the specific combination between P(x) and P(y|x). The error can be higher, lower or the same. A partial model reconstruction or a retraining using new P(x) can improve accuracy. Conditional change. The conditional probability P(y|x) changes. Some feature vectors previously classified as y are now classified differently. It is likely a completely new model extraction. Dual change. Both P(x) and P(y|x) change. Since P(y|x) has evolved, it is necessary to train a completely new model. Generally, without any assumption about data distribution, in none of the aforementioned four cases, the expected error can be used as an indication of concept drifting. Therefore, it is very difficult to find a general correlation between the expected error of the previously extracted model and any type of concept drifting. Even though the expected error of a model does not change, a variation in the distribution can occur and the classifier is not updated to the current data. 2.2 Related work Data streams processing has recently become an important research domain. This aspect is testified by the number of papers currently published and by the real applications readily

View PDFchevron_right
Challenges in Data Stream Classification
Dr. Jagannath E . Nalavade

2015

Stream data mining is emerging field in the data mining. Stream data generates from many applications such as banking, sensor networks, blogs at twitter. It is conceptually endless sequences of data records. It is often arriving at high rates. Analyzing or mining data streams raises several new issues compared to standard data mining algorithms. Standard data mining algorithms assume that records can be examined multiple times. Data stream mining algorithms, on the other hand, are more challenging to design since they must be able to extract all necessary information from records with only one pass over the data. Data stream mining algorithms must be online. Arrival rates for records are high, so the practical complexity of processing must also be low. Since the classification algorithm executes endlessly, it must be able to adapt the classification model to changes in the data stream, in particular to changes in the boundaries between classes (“concept drift”). To maximize usefulne...

View PDFchevron_right
A Survey on Supervised Classification on Data Streams
Vincent Lemaire

Lecture Notes in Business Information Processing, 2015

The last ten years were prolific in the statistical learning and data mining field and it is now easy to find learning algorithms which are fast and automatic. Historically a strong hypothesis was that all examples were available or can be loaded into memory so that learning algorithms can use them straight away. But recently new use cases generating lots of data came up as for example: monitoring of telecommunication network, user modeling in dynamic social network, web mining... The volume of data increases rapidly and it is now necessary to use incremental learning algorithms on data streams. This article presents the main approaches of incremental supervised classification available in the literature. It aims to give basic knowledge to a reader novice in this subject.

View PDFchevron_right
A Survey on Issues of Data Stream Mining in Classification
Nirav Bhatt

2017

As Data Stream Mining is trending topic for Research nowadays and more users increases day by day with online stuff, the size of big data is also getting larger. In traditional data mining extracting knowledge is done mostly using offline phase. While in data stream, Extracting data is from the continuous arriving data or we can say from the online streams. Due to continuously arriving data, it cannot be stored in the memory for processing permanently. So examining of data as fast as possible is important. In this paper we would be interested to discuss about the data stream mining and the issues of stream classification, like Single scan, Load shedding, Memory Space, Class imbalance problem, Concept drift, and possible ways to solve those issues.

View PDFchevron_right

Related topics

  • Random Forests

    • Cited by

    Classifying Evolving Data Streams Using Dynamic Streaming Random Forests
    hanady abdulsalam

    Lecture Notes in Computer Science, 2008

    We consider the problem of data-stream classification, introducing a stream-classification algorithm, Dynamic Streaming Random Forests, that is able to handle evolving data streams using an entropy-based drift-detection technique. The algorithm automatically adjusts its parameters based on the data seen so far. Experimental results show that the algorithm handles multi-class problems for which the underlying class boundaries drift, without losing accuracy.

    View PDFchevron_right
    A Novel Drift Detection Algorithm Based on Features’ Importance Analysis in a Data Streams Environment
    Krzysztof Przybyszewski

    Journal of Artificial Intelligence and Soft Computing Research, 2020

    The training set consists of many features that influence the classifier in different degrees. Choosing the most important features and rejecting those that do not carry relevant information is of great importance to the operating of the learned model. In the case of data streams, the importance of the features may additionally change over time. Such changes affect the performance of the classifier but can also be an important indicator of occurring concept-drift. In this work, we propose a new algorithm for data streams classification, called Random Forest with Features Importance (RFFI), which uses the measure of features importance as a drift detector. The RFFT algorithm implements solutions inspired by the Random Forest algorithm to the data stream scenarios. The proposed algorithm combines the ability of ensemble methods for handling slow changes in a data stream with a new method for detecting concept drift occurrence. The work contains an experimental analysis of the proposed...

    View PDFchevron_right
    Interactive generic learning method (IGLM)
    Christel Pallez

    2016

    We propose a method to improve the performance of Random Forests for classifying short texts interactively. In short text classification, the principle of learning algorithms is to build a static model using a training dataset, then to use this model to classify new texts. Many works concentrate on improving the representation of the data as a way to build better models. We intend to tackle the problem in two ways: first by abstracting data to solve the problem of sparseness, and second by taking benefit from already classified data to continuously improve the model. Besides, in order to alleviate the amount of manual annotation, we propose an interactive method in which a manual correct annotation is required only for some misclassified texts, which are then incorporated into the training data to build an updated model. An important challenge is then to determine when to trigger this operation and how to perform the update. Applied on the standard search-snippets dataset, our method allowed a significant improvement.

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved