Academia.eduAcademia.edu

Figure 6 – uploaded by Dejan Gjorgjevikj

See full PDFdownloadDownload figure
Fig. 6. The critical diagrams for the ranking-based evaluation measures: the results from the Nemenyi post-hoc test at 0.05 significance level on the datasets for which all algorithms provided results: (a) ranking loss; (b) one-error; (c) coverage; and (d) average precision.

Figure 6 The critical diagrams for the ranking-based evaluation measures: the results from the Nemenyi post-hoc test at 0.05 significance level on the datasets for which all algorithms provided results: (a) ranking loss; (b) one-error; (c) coverage; and (d) average precision.

Related Figures (30)

Fig. 2. The multi-label learning methods used in this study divided into groups based on the base machine learning algorithm they use. Fig. 1. The multi-label learning methods used in this study divided into groups as discussed in the related work section above.
Fig. 2. The multi-label learning methods used in this study divided into groups based on the base machine learning algorithm they use. Fig. 1. The multi-label learning methods used in this study divided into groups as discussed in the related work section above.
Description of the benchmark problems in terms of application domain (domain), number of training (#tr.e.) and test (#t.e.) examples, the number of features (D), the total number of labels (Q) and label cardinality (I.). The problems are ordered by their overall complexity roughly calculated as #tr.e.x Dx Q where each instance is a piece of music. Each piece of music can be labeled with six emotions: sad-lonely, angry-aggressive, amazed-surprised, relaxing-calm, quiet-still, and happy-pleased. Scene [32] is a widely used scene classification dataset. Each scene can be annotated in the following six contexts: beach, sunset, field, fall-foliage, mountain, and urban. The Corel5k [34] data set contains Corel images that are segmented using normalized cuts. The segmented regions are then clustered into 499 bins, which are further used to describe the images. Each image can be then assigned several of the 374 possible labels. Mediamill [36] origi- nates from the 2005 NIST TRECVID challenge dataset,” which contains data about annotated videos. The label space is repre- sented by 101 “annotation concepts”, such as explosion, aircraft, face, truck, urban, etc.
Description of the benchmark problems in terms of application domain (domain), number of training (#tr.e.) and test (#t.e.) examples, the number of features (D), the total number of labels (Q) and label cardinality (I.). The problems are ordered by their overall complexity roughly calculated as #tr.e.x Dx Q where each instance is a piece of music. Each piece of music can be labeled with six emotions: sad-lonely, angry-aggressive, amazed-surprised, relaxing-calm, quiet-still, and happy-pleased. Scene [32] is a widely used scene classification dataset. Each scene can be annotated in the following six contexts: beach, sunset, field, fall-foliage, mountain, and urban. The Corel5k [34] data set contains Corel images that are segmented using normalized cuts. The segmented regions are then clustered into 499 bins, which are further used to describe the images. Each image can be then assigned several of the 374 possible labels. Mediamill [36] origi- nates from the 2005 NIST TRECVID challenge dataset,” which contains data about annotated videos. The label space is repre- sented by 101 “annotation concepts”, such as explosion, aircraft, face, truck, urban, etc.
Fig. 3. Categorization of evaluation measures used to assess the predictive performance of methods for multi-label learning. The ranking-based evaluation measures compare the predicted ranking of the labels with the ground truth ranking. We used four ranking-based measures: one-error, coverage, ranking loss and average precision. A detailed description of the evaluation mea- sures is given in Appendix A.
Fig. 3. Categorization of evaluation measures used to assess the predictive performance of methods for multi-label learning. The ranking-based evaluation measures compare the predicted ranking of the labels with the ground truth ranking. We used four ranking-based measures: one-error, coverage, ranking loss and average precision. A detailed description of the evaluation mea- sures is given in Appendix A.
Fig. 4. The critical diagrams for the example-based evaluation measures: the results from the Nemenyi post-hoc test at 0.05 significance level on the datasets for which al algorithms provided results. For precision and subset accuracy the differences are not statistically significant according to the Friedman test (see Table B7), thus we shov only the average ranks of the algorithms: (a) Hamming loss; (b) accuracy; (c) precision; (d) recall; (e) subset accuracy; and (f) F; score.
Fig. 4. The critical diagrams for the example-based evaluation measures: the results from the Nemenyi post-hoc test at 0.05 significance level on the datasets for which al algorithms provided results. For precision and subset accuracy the differences are not statistically significant according to the Friedman test (see Table B7), thus we shov only the average ranks of the algorithms: (a) Hamming loss; (b) accuracy; (c) precision; (d) recall; (e) subset accuracy; and (f) F; score.
Fig. 5. The critical diagrams for the label-based evaluation measures: the results from the Nemenyi post-hoc test at 0.05 significance level on the datasets for which all algorithms provided results: (a) micro-precision; (b) macro-precision; (c) micro-recall; (d) macro-recall; (e) micro-F,; and (f) macro-F;.
Fig. 5. The critical diagrams for the label-based evaluation measures: the results from the Nemenyi post-hoc test at 0.05 significance level on the datasets for which all algorithms provided results: (a) micro-precision; (b) macro-precision; (c) micro-recall; (d) macro-recall; (e) micro-F,; and (f) macro-F;.
Fig. 7. The critical diagrams for the efficiency measures: the results from the Nemenyi post-hoc test at 0.05 significance level on the datasets for which all algorithm: provided results: (a) training time and (b) testing time.
Fig. 7. The critical diagrams for the efficiency measures: the results from the Nemenyi post-hoc test at 0.05 significance level on the datasets for which all algorithm: provided results: (a) training time and (b) testing time.
The performance of the multi-label learning approaches in terms of the accuracy measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources. Table B2
The performance of the multi-label learning approaches in terms of the accuracy measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources. Table B2
The performance of the multi-label learning approaches in terms of the Hamming loss measure. DNF (Did Not Finish) stands for the algorithms that did not construct < predictive model within one week under the available resources. Table B1
The performance of the multi-label learning approaches in terms of the Hamming loss measure. DNF (Did Not Finish) stands for the algorithms that did not construct < predictive model within one week under the available resources. Table B1
The performance of the multi-label learning approaches in terms of the recall measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the recall measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the precision measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the precision measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the F; score measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the F; score measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the subset accuracy measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the subset accuracy measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the micro-precision measure. DNF (Did Not Finish) stands for the algorithms that did not construct « predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the micro-precision measure. DNF (Did Not Finish) stands for the algorithms that did not construct « predictive model within one week under the available resources.
The p-values of the assessment of performance of the multi-label learning approaches by the Friedman test using the example-based evaluation measures. Subset shows the calculated p-values for the datasets on which all algorithms finished. All shows the calculated p-values for all datasets including those which did not finished. Fig. B1. The critical diagrams for the example-based evaluation measures: the results from the Nemenyi post-hoc test at 0.05 significance level on all the datasets. Fo Hamming loss, precision, accuracy and subset accuracy the differences are not statistically significant according to the Friedman test (see Table B7), thus we show only th average ranks of the algorithms: (a) Hamming loss; (b) accuracy; (c) precision; (d) recall; (e) subset accuracy; and (f) F; score.
The p-values of the assessment of performance of the multi-label learning approaches by the Friedman test using the example-based evaluation measures. Subset shows the calculated p-values for the datasets on which all algorithms finished. All shows the calculated p-values for all datasets including those which did not finished. Fig. B1. The critical diagrams for the example-based evaluation measures: the results from the Nemenyi post-hoc test at 0.05 significance level on all the datasets. Fo Hamming loss, precision, accuracy and subset accuracy the differences are not statistically significant according to the Friedman test (see Table B7), thus we show only th average ranks of the algorithms: (a) Hamming loss; (b) accuracy; (c) precision; (d) recall; (e) subset accuracy; and (f) F; score.
The performance of the multi-label learning approaches in terms of the macro-precision measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the macro-precision measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the macro-recall measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the macro-recall measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the micro-recall measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the micro-recall measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the micro-F; measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the micro-F; measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the macro-F; measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the macro-F; measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the ranking loss measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the ranking loss measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the one-error measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the one-error measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The p-values of the assessment of performance of the multi-label learning approaches by the Friedman test using the label-based evaluation measures. Subset shows the calculated p-values for the datasets on which all algorithms finished. All shows the calculated p-values for all datasets including those which did not finished.
The p-values of the assessment of performance of the multi-label learning approaches by the Friedman test using the label-based evaluation measures. Subset shows the calculated p-values for the datasets on which all algorithms finished. All shows the calculated p-values for all datasets including those which did not finished.
The performance of the multi-label learning approaches in terms of the coverage measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources. Table B17
The performance of the multi-label learning approaches in terms of the coverage measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources. Table B17
Fig. B3. The critical diagrams for the ranking-based evaluation measures: the results from the Nemenyi post-hoc test at 0.05 significance level on all the datasets: (a) ranking loss; (b) one-error; (c) coverage; and (d) average precision.
Fig. B3. The critical diagrams for the ranking-based evaluation measures: the results from the Nemenyi post-hoc test at 0.05 significance level on all the datasets: (a) ranking loss; (b) one-error; (c) coverage; and (d) average precision.
The performance of the multi-label learning approaches in terms of the average precision measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the average precision measure. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources.
The p-values of the assessment of performance of the multi-label learning approaches by the Friedman test using the ranking-based evaluation measures. Subset shows the calculated p-values for the datasets on which all algorithms finished. All shows the calculated p-values for all datasets including those which did not finished.
The p-values of the assessment of performance of the multi-label learning approaches by the Friedman test using the ranking-based evaluation measures. Subset shows the calculated p-values for the datasets on which all algorithms finished. All shows the calculated p-values for all datasets including those which did not finished.
The performance of the multi-label learning approaches in terms of the testing time measures in seconds. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources).
The performance of the multi-label learning approaches in terms of the testing time measures in seconds. DNF (Did Not Finish) stands for the algorithms that did not construct a predictive model within one week under the available resources).
The performance of the multi-label learning approaches in terms of the training time measures in seconds. DNF (Did Not Finish) stands for the algorithms that did no construct a predictive model within one week under the available resources.
The performance of the multi-label learning approaches in terms of the training time measures in seconds. DNF (Did Not Finish) stands for the algorithms that did no construct a predictive model within one week under the available resources.
Related topics:
Classification (Machine Learning)Multi-label learningMulti-Label Classification
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved