Improving Stability of Decision Trees

2013

Neural Networks and Soft Computing, 2003

In many cases it is better to extract a set of decision trees and a set of possible logical data descriptions instead of a single model. Methods for creating forests of decision trees based on Separability of Split Value (SSV) criterion are presented. Preliminary results confirm their usefulness in understanding data structures.

International Journal of Fuzzy System Applications

Fuzzy decision tree (FDT) is a powerful top-down, hierarchical search methodology to extract human interpretable classification rules. The performance of FDT depends on initial fuzzy partitions and other parameters like alpha-cut and leaf selection threshold. These parameters are decided either heuristically or by trial-and-error. For given set of parameters, FDT is constructed using any standard induction algorithms like Fuzzy ID3. Due to the greedy nature of induction process, there is a chance of FDT resulting in poor classification accuracy. To further improve the accuracy of FDT, in this paper, the authors propose the strategy called Improved Second Order- Neuro- Fuzzy Decision Tree (ISO-N-FDT). ISO-N-FDT tunes parameters of FDT from leaf node to roof node starting from left side of tree to its right and attains better improvement in accuracy with less number of iterations exhibiting fast convergence and powerful search ability.

entropy, 2010

Crisp Decision trees (CDT) algorithms have been the most widely employed methodologies for symbolic knowledge acquisition. There are many methodologies have been presented to address the problems of the continuous data, multi-valued data, missing data, uncertainty data and noisy features. Recently, due to the widespread use of the fuzzy representation, a lot of researchers have utilized the fuzzy representation in decision trees to overcome the preceding problems. Fuzzy decision trees (FDT) are generalization for the CDT. FDTs are built by using fuzzy or crisp attributes and classes which often need pruning to reduce their size. FDTs have been successfully used to extract knowledge in uncertain classification problems. In this paper, we present a technique to build FDT by employing the ambiguity of attributes and classifiability of instance. Our technique builds a reduced FDT which does not need for applying the pruning algorithms to reduce the size. The paper also presents the results of a set of empirical studies conducted on a dataset of UCI Repository of Machine Learning Database that evaluate the effectiveness of our technique compared to Fussy Iterative Dichotomiser 3 (FID3), ambiguity, and FID3 with classifiability techniques. The studies show the effective of our technique in reducing the number of the extracted rules without loosing of the rules accuracy.

Data Mining and Knowledge Discovery, 1998

Decision trees have proved to be valuable tools for the description, classification and generalization of data. Work on constructing decision trees from data exists in multiple disciplines such as statistics, pattern recognition, decision theory, signal processing, machine learning and artificial neural networks. Researchers in these disciplines, sometimes working on quite different problems, identified similar issues and heuristics for decision tree construction. This paper surveys existing work on decision tree construction, attempting to identify the important issues involved, directions the work has taken and the current state of the art.

Information Sciences, 2005

Label semantics is a random set based framework for ''Computing with Words'' that captures the idea of computation on linguistic terms rather than numerical quantities. Within this new framework, a decision tree learning model is proposed where nodes are linguistic descriptions of variables and leaves are sets of appropriate labels. In such decision trees, the probability estimates for branches across the whole tree is used for classification, instead of the majority class of the single branch into which the examples fall. By empirical experiments on real-world datasets it is verified that our algorithm has better or equivalent classification accuracy compared to three well known machine learning algorithms. By applying a new forward branch merging algorithm, the complexity of the tree can be greatly reduced without significant loss of accuracy. Finally, a linguistic interpretation of trees and classification with linguistic constraints are introduced.

2019

Data mining and machine learning (ML) are increasingly at the core of many aspects of modern life. With growing concerns about the impact of relying on predictions we cannot understand, there is widespread agreement regarding the need for reliable interpretable models. One of the areas where this is particularly important is clinical decision-making. Specifically, explainable models have the potential to facilitate the elaboration of clinical guidelines and related decision-support tools. The presented research focuses on the improvement of decision tree (DT) learning, one of the most popular interpretable models, motivated by the challenges posed by clinical data. One of the limitations of interpretable DT algorithms is that they involve decisions based on strict thresholds, which can impair performance in the presence noisy measurements. In this regard, we proposed a probabilistic method that takes into account a model of the noise in the distinct learning phases. When considering...

Proceedings of the 2016 Conference on Information Technologies in Science, Management, Social Sphere and Medicine, 2016

Decision trees have been recognized as interpretable, efficient, problem independent and scalable architectures. In case of fuzzy representation there is no procedure of automation tree building. In other words existing approaches of building decision trees and fuzzy decision trees cannot provide automatically generate fuzzy sets and fuzzy knowledge bases to build fuzzy decision trees. Paper presents a new method of building fuzzy decision trees called decision trees based fuzzy rules (DTFR). This method combines tree growing and pruning, to determine the structure of the FDT, to improve its generalization capabilities. Proposes a method (DTFR) considered as a variant of decision tree inductive using fuzzy set theory.

2014

Over the years, various methodologies have been investigated and proposed to deal with continuous or numeric data which is very common in any application. With the increasing popularity of fuzzy representation, researchers have proposed to utilize fuzzy logic in decision trees to deal with the situations. This paper presents a survey of current methodology to design FDT (Fuzzy Decision Tree), various issues and applications. The author conclude that fuzzy decision making using decision tree is an emerging technique in terms of applications and there is a enough scope of research in this area. KeywordsContinuous data, Decision tree, Fuzzy logic, Fuzzy decision tree, numeric data,

International Journal of Advanced Research in Computer Science and Software Engineering , 2013

Classic decision sapling classifiers assist data whose values are usually known along with precise. We extend such classifiers to manage data using uncertain information .Value scepticism arises in numerous applications throughout the data collection process. Example reasons for uncertainty consist of measurement/quantization problems, data staleness, along with multiple repeated measurements. Along with uncertainty, the worth of any data item is normally represented certainly not by a unitary value, yet by numerous values being created a likelihood distribution. As opposed to abstracting unsure data by means of statistical derivatives (such while mean along with median), we see that the accuracy of a decision sapling classifier is usually much improved if the "complete information" of a data item (taking into account the likelihood density operate (pdf)) is utilized. We extend classical conclusion tree constructing algorithms to manage data tuples using uncertain prices. Extensive experiments have been conducted which usually show that this resulting classifiers will be more accurate than those utilizing value averages. I. INTRODUCTION Classification is a classical problem in machine learning and data mining. Given a set of training data tuples, each having a class label and being represented by a feature vector, the task is to algorithmically build a model that predicts the class label of an unseen test tuple based on the tuple's feature vector. One of the most popular classification models is the decision tree model. Decision trees are popular because they are practical and easy to understand. Rules can also be extracted from decision trees easily. Many algorithms, such as ID3 and C4.5, have been devised for decision tree construction. These algorithms are widely adopted and used in a wide range of applications such as image recognition, medical diagnosis, credit rating of loan applicants, scientific tests, fraud detection, and target marketing. In traditional decision tree classification, a feature (an attribute) of a tuple is either categorical or numerical. For the latter, a precise and definite point value is usually assumed. In many applications, however, data uncertainty is common. The value of a feature/attribute is thus best captured not by a single point value, but by a range of values giving rise to a probability distribution. A simple way to handle data uncertainty is to abstract probability distributions by summary statistics such as means and variances. We call this approach averaging. Another approach is to consider the complete information carried by the probability distributions to build a decision tree. We call this approach Distribution-based. In this paper, we study the problem of constructing decision tree classifiers on data with uncertain numerical attributes. Our goals are 1) to devise an algorithm for building decision trees from uncertain data using the Distribution-based approach, 2) to investigate whether the Distribution-based approach could lead to a higher classification accuracy compared with the Averaging approach, and 3) to establish a theoretical foundation on which pruning techniques are derived that can significantly improve the computational efficiency of the Distribution-based algorithms.