Academia.eduAcademia.edu

Figure 2 – uploaded by Mahesh Pal

See full PDFdownloadDownload figure
Fig. 2. Axis-parallel decision boundaries of a univariate decision tree.

Figure 2 Axis-parallel decision boundaries of a univariate decision tree.

Related Figures (11)

Fig. 1. A classification tree with four dimensional feature space and three classes. The x; are feature values; a, b, c, d, and e are the thresholds and A, B, and C are class labels. DT construction involves the recursive partitioning of a set of training data, which is split into increasingly homo- geneous subsets on the basis of tests applied to one or more of the feature values. These tests are represented by nodes. The univariate DT applies a test to a single feature at a time, whereas the multivariate DT uses one or more features simultaneously. Labels are assigned to terminal (leaf) nodes by means of an allocation strategy, such as majority voting. At one time, DTs were designed manually, using spectral plots. In the past decade, automatic methods of decision tree design have been developed. In this study, two univariate DT algorithms, C4.5 and See5.0 (Quinlan, 1993, 1996), and
Fig. 1. A classification tree with four dimensional feature space and three classes. The x; are feature values; a, b, c, d, and e are the thresholds and A, B, and C are class labels. DT construction involves the recursive partitioning of a set of training data, which is split into increasingly homo- geneous subsets on the basis of tests applied to one or more of the feature values. These tests are represented by nodes. The univariate DT applies a test to a single feature at a time, whereas the multivariate DT uses one or more features simultaneously. Labels are assigned to terminal (leaf) nodes by means of an allocation strategy, such as majority voting. At one time, DTs were designed manually, using spectral plots. In the past decade, automatic methods of decision tree design have been developed. In this study, two univariate DT algorithms, C4.5 and See5.0 (Quinlan, 1993, 1996), and
Fig. 3. Decision boundaries for a multivariate decision tree classifier.
Fig. 3. Decision boundaries for a multivariate decision tree classifier.
Fig. 4. Variation of classification accuracy with increasing number of training patterns using univariate decision tree classifier and ETM + data (test data set 1, Littleport).
Fig. 4. Variation of classification accuracy with increasing number of training patterns using univariate decision tree classifier and ETM + data (test data set 1, Littleport).
Fig. 5. Variation of classification accuracy with increasing number of training patterns using a multivariate decision tree classifier and ETM + data (test data set 1, Littleport).
Fig. 5. Variation of classification accuracy with increasing number of training patterns using a multivariate decision tree classifier and ETM + data (test data set 1, Littleport).
Fig. 6. Classification accuracies using DAIS hyperspectral data (test data set 2, La Mancha) with fixed training set of 250 pixels/class and increasing number of features.
Fig. 6. Classification accuracies using DAIS hyperspectral data (test data set 2, La Mancha) with fixed training set of 250 pixels/class and increasing number of features.
Fig. 7. Variation of accuracy with different attribute selection measures using ETM + data (test data set 1, Littleport) with 2700 training pixels. Simplification involves the removal of those parts of the tree that do not contribute to classification accuracy on unseen cases, thus producing a less complex and more
Fig. 7. Variation of accuracy with different attribute selection measures using ETM + data (test data set 1, Littleport) with 2700 training pixels. Simplification involves the removal of those parts of the tree that do not contribute to classification accuracy on unseen cases, thus producing a less complex and more
The performance of the REP method is worst, with a classification accuracy of 81.4%. The reason for this could be the requirement of a separate data set for pruning, a conclusion also suggested by Esposito, Malerba, and Semer- aro (1997). PEP gives the highest accuracy of 82.9%;
The performance of the REP method is worst, with a classification accuracy of 81.4%. The reason for this could be the requirement of a separate data set for pruning, a conclusion also suggested by Esposito, Malerba, and Semer- aro (1997). PEP gives the highest accuracy of 82.9%;
Results from maximum likelihood and neural network classifiers using ETM + data (test data set 1) with 2700 training pixels and seven classes however, Esposito et al. (1997) note that the introduction of a continuity correction in the estimation of error rate has no theoretical justification and such a factor is improperly compared to an error rate, which may lead to either under- pruning or overpruning of the tree. The performance of CVP is affected by the choice of critical value set to prune a tree. CCP uses a separate data set or a cross-validation approach to pruning, and a pruned sub-tree is selected by minimising a complexity factor over a set of pruned sub-trees. Gelfand, Ravishankar, and Delp (1991) suggest that this set of sub- trees may not include the optimum sub-tree. Finally, EBP uses training data for pruning the tree. These results suggest that the choice of a suitable pruning method is an important factor in the design of a DT classifier in comparison to the attribute selection measures, as the availability of a suffi- cient number of training data elements is always a problem. Our conclusion is that EBP, which gives an overall classi- fication accuracy of 82.8%, is the preferred method. This conclusion is not based exclusively on the relatively small increase in the level of classification accuracy that results from the use of pruning; it also takes into account the benefits deriving from the reduction in tree size and the simplification of the tree, which can improve interpretation and understanding.
Results from maximum likelihood and neural network classifiers using ETM + data (test data set 1) with 2700 training pixels and seven classes however, Esposito et al. (1997) note that the introduction of a continuity correction in the estimation of error rate has no theoretical justification and such a factor is improperly compared to an error rate, which may lead to either under- pruning or overpruning of the tree. The performance of CVP is affected by the choice of critical value set to prune a tree. CCP uses a separate data set or a cross-validation approach to pruning, and a pruned sub-tree is selected by minimising a complexity factor over a set of pruned sub-trees. Gelfand, Ravishankar, and Delp (1991) suggest that this set of sub- trees may not include the optimum sub-tree. Finally, EBP uses training data for pruning the tree. These results suggest that the choice of a suitable pruning method is an important factor in the design of a DT classifier in comparison to the attribute selection measures, as the availability of a suffi- cient number of training data elements is always a problem. Our conclusion is that EBP, which gives an overall classi- fication accuracy of 82.8%, is the preferred method. This conclusion is not based exclusively on the relatively small increase in the level of classification accuracy that results from the use of pruning; it also takes into account the benefits deriving from the reduction in tree size and the simplification of the tree, which can improve interpretation and understanding.
Fig. 9. Classification accuracies using DAIS hyperspectral data (test data set 2, La Mancha) with a fixed training set of 250 pixels/class and an increasin number of features. The classification accuracies obtained with a boosted univariate decision tree are also shown. Boosting is a method of improving the performance of a “weak”? classifier. In essence, this improvement is achieved Classification accuracies and Kappa values obtained from unboosted and boosted DTs, estimated using sets of 2700 training and 2037 separate test data for test area | (Littleport) are shown in Table 2. The boosted DT classi- fications were estimated using 14 iterations of the base decision tree algorithm. The number of boosting iterations was varied from 2 to 20 but there was little change in classification accuracy beyond 14 iterations, and the degree of accuracy improvement achieved through the use of boosting starts to stabilise after eight iterations. It appears that 10-15 boosting iterations are sufficient to achieve an improvement in classification accuracy for this type of data. The result also concurs with the conclusions of studies using non-remote sensing data. Quinlan (1996) concludes that about 10 iterations is the optimum number, and that little is gained by performing additional boosting runs. We noted an increase of more than 4% in the level of classification accuracy following boosting, using test data set 1, which is a small increase in comparison to the results reported by Quinlan (1996) and Muchoney et al. (2000). Although the level of improvement is small, it should be borne in mind that even small percentage increases are difficult to generate when the overall classification accuracy level exceeds 80%.
Fig. 9. Classification accuracies using DAIS hyperspectral data (test data set 2, La Mancha) with a fixed training set of 250 pixels/class and an increasin number of features. The classification accuracies obtained with a boosted univariate decision tree are also shown. Boosting is a method of improving the performance of a “weak”? classifier. In essence, this improvement is achieved Classification accuracies and Kappa values obtained from unboosted and boosted DTs, estimated using sets of 2700 training and 2037 separate test data for test area | (Littleport) are shown in Table 2. The boosted DT classi- fications were estimated using 14 iterations of the base decision tree algorithm. The number of boosting iterations was varied from 2 to 20 but there was little change in classification accuracy beyond 14 iterations, and the degree of accuracy improvement achieved through the use of boosting starts to stabilise after eight iterations. It appears that 10-15 boosting iterations are sufficient to achieve an improvement in classification accuracy for this type of data. The result also concurs with the conclusions of studies using non-remote sensing data. Quinlan (1996) concludes that about 10 iterations is the optimum number, and that little is gained by performing additional boosting runs. We noted an increase of more than 4% in the level of classification accuracy following boosting, using test data set 1, which is a small increase in comparison to the results reported by Quinlan (1996) and Muchoney et al. (2000). Although the level of improvement is small, it should be borne in mind that even small percentage increases are difficult to generate when the overall classification accuracy level exceeds 80%.
Shaded values indicate improvements in the performance of first-named classifier at the 95% confidence level (critical value of Z= 1.96). Unshaded value indicates that both classifiers perform equally well. WB means “without boosting” and B means “boosting” a decision tree classifier. Calculated Z values for comparison between different classification systems Table 3
Shaded values indicate improvements in the performance of first-named classifier at the 95% confidence level (critical value of Z= 1.96). Unshaded value indicates that both classifiers perform equally well. WB means “without boosting” and B means “boosting” a decision tree classifier. Calculated Z values for comparison between different classification systems Table 3
Classification accuracies achieved using 2000 training and 3800 test data with all 65 features of the hyperspectral data set (test data set 2, La Mancha) Table 4 found not to be important. The use of boosting is recom- mended; in this study, it resulted in an improvement in classification accuracy of about 3—4%, at little cost in computer time or complexity of use.
Classification accuracies achieved using 2000 training and 3800 test data with all 65 features of the hyperspectral data set (test data set 2, La Mancha) Table 4 found not to be important. The use of boosting is recom- mended; in this study, it resulted in an improvement in classification accuracy of about 3—4%, at little cost in computer time or complexity of use.
Related topics:
AlgorithmsHigh Dimensional DataGeomatic EngineeringLand cover classificationTHERMAL INFRARED REMOTE SENSING DATAAttribute Selection
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved