Academia.eduAcademia.edu

Figure 11 – uploaded by Domenico Ciuonzo

See full PDFdownloadDownload figure
(d) Class-Indifferent (CI): Decision Templates (DT) and Dempster-Shafer (DS) approaches

Table 11 (d) Class-Indifferent (CI): Decision Templates (DT) and Dempster-Shafer (DS) approaches

Related Figures (17)

Figure 1: Architecture of the Multi-Classification System (MCS) proposed. are later provided in Sec.|5) to the hard/soft combiner. We briefly describe their main properties and the motivations that cuided us to their choice. For all of them we have reproduced their exact implementation and executed them with the same parameters as described in the respective works, to which we refer for further details. from the sequence of packets forming it. It is worth mention- ing that other feature sets may be considered, especially when early-TC of the generic SB is deemed of interest. The afore- mentioned class includes the packet sizes of the first K packets or some statistical features extracted from this "early" segment, as studied in for the case of Internet TC.
Figure 1: Architecture of the Multi-Classification System (MCS) proposed. are later provided in Sec.|5) to the hard/soft combiner. We briefly describe their main properties and the motivations that cuided us to their choice. For all of them we have reproduced their exact implementation and executed them with the same parameters as described in the respective works, to which we refer for further details. from the sequence of packets forming it. It is worth mention- ing that other feature sets may be considered, especially when early-TC of the generic SB is deemed of interest. The afore- mentioned class includes the packet sizes of the first K packets or some statistical features extracted from this "early" segment, as studied in for the case of Internet TC.
Table 1: Summary of state-of-art techniques selected as base classifiers. with categorical and numerical target variables, respectively). The above classifier is fed with the same statistical features as Tay_RF and Tay_SVC. sizes Matcn best with the observed values in the test instance, the MNB classifier treats the f,,s as frequencies of a certain value of a categorical random variable and compares the sample histogram of each test instance with the aggregated histogram of all training instances per class. Then, the evaluation of the conditional PMF P(f7|c;) is different from Lib_NB and equals P(frlci) « TM Om), where p,, denotes the probability of sampling the m'” feature. This implementation is referred to as Her_Pure in our analysis. A few variants of MNB classifier, adopting term frequency transformation without and with co- sine normalization, were also successfully employed in [/7] and compared in , and are referred in our analysis to as Her_TF and Her_Cos, respectively.
Table 1: Summary of state-of-art techniques selected as base classifiers. with categorical and numerical target variables, respectively). The above classifier is fed with the same statistical features as Tay_RF and Tay_SVC. sizes Matcn best with the observed values in the test instance, the MNB classifier treats the f,,s as frequencies of a certain value of a categorical random variable and compares the sample histogram of each test instance with the aggregated histogram of all training instances per class. Then, the evaluation of the conditional PMF P(f7|c;) is different from Lib_NB and equals P(frlci) « TM Om), where p,, denotes the probability of sampling the m'” feature. This implementation is referred to as Her_Pure in our analysis. A few variants of MNB classifier, adopting term frequency transformation without and with co- sine normalization, were also successfully employed in [/7] and compared in , and are referred in our analysis to as Her_TF and Her_Cos, respectively.
Figure 3: Accuracy (a-c) and F-measure (d-f) of the base (state-of-the-art) classifiers to varying the BT (Android dataset). Figure 2: Number of service bursts for different values of the BT considering Android (a) and iOS (b) datasets.
Figure 3: Accuracy (a-c) and F-measure (d-f) of the base (state-of-the-art) classifiers to varying the BT (Android dataset). Figure 2: Number of service bursts for different values of the BT considering Android (a) and iOS (b) datasets.
Figure 4: Accuracy (a) and F-measure (b) of the best base classifier, hard and soft combiner versus the BT (Android dataset). Performance refers to the dataset with zero-payload packets removed before the SB extraction.
Figure 4: Accuracy (a) and F-measure (b) of the best base classifier, hard and soft combiner versus the BT (Android dataset). Performance refers to the dataset with zero-payload packets removed before the SB extraction.
Table 2: Performance (%) of base (state-of-the-art) classifiers considering Android (iOS) traffic. Table 3: Performance (%) of hard combiners considering Android (iOS) traffic.
Table 2: Performance (%) of base (state-of-the-art) classifiers considering Android (iOS) traffic. Table 3: Performance (%) of hard combiners considering Android (iOS) traffic.
each group of (hard/soft) combining techniques, we have re- ported in Tab. 5|the Maximum Improvement Over the Best Clas- sifier (MIOBC) provided by each group for every performance measure. Referring to the group of hard combiners, it is ap- parent that such group is always able to provide an improve- ment, ranging from +2.2% (accuracy on Android traffic) to +6.6% (recall on Android traffic), by means of diversity prin- ciple, representing the milestone for adoption of classifier fu- sion techniques. However, as remarked before, different ap- proaches (namely MV, WMV, NB, and BKS) result best according to different performance metrics. A similar reasoning applies to the group of CC non-trainable combiners (second column), where the improvement ranges from +3.0% (accuracy on An- droid traffic) to +6.5% (recall on Android traffic). Here, the im- provement is qualitatively similar to hard combiners. However, CC non-trainable combiners, though requiring the availability of soft outputs from each classifier in the pool, do not require the availability of a validation set (which is instead required in the design of almost all the hard combiners here considered). This may be appealing in the case of scarcity of additional train- ing (validation) data. On the other hand, the group of CC train- able combiners is able to provide the best improvement for each metric. This is not only the consequence of the presence of KL weights within the group, having the highest performance. In- deed, as observed earlier, all the combiners within the group are able to provide significant gains. Finally, CI combiners are able to collectively provide a performance improvement over almost all the considered metrics. The sole exception is represented by precision (which is slightly degraded for all the combiners within this group), with a consequent gain reduction of the cor- responding highest F-measure achieved by the CI group. This is more evident when a soft combiner is employed.
each group of (hard/soft) combining techniques, we have re- ported in Tab. 5|the Maximum Improvement Over the Best Clas- sifier (MIOBC) provided by each group for every performance measure. Referring to the group of hard combiners, it is ap- parent that such group is always able to provide an improve- ment, ranging from +2.2% (accuracy on Android traffic) to +6.6% (recall on Android traffic), by means of diversity prin- ciple, representing the milestone for adoption of classifier fu- sion techniques. However, as remarked before, different ap- proaches (namely MV, WMV, NB, and BKS) result best according to different performance metrics. A similar reasoning applies to the group of CC non-trainable combiners (second column), where the improvement ranges from +3.0% (accuracy on An- droid traffic) to +6.5% (recall on Android traffic). Here, the im- provement is qualitatively similar to hard combiners. However, CC non-trainable combiners, though requiring the availability of soft outputs from each classifier in the pool, do not require the availability of a validation set (which is instead required in the design of almost all the hard combiners here considered). This may be appealing in the case of scarcity of additional train- ing (validation) data. On the other hand, the group of CC train- able combiners is able to provide the best improvement for each metric. This is not only the consequence of the presence of KL weights within the group, having the highest performance. In- deed, as observed earlier, all the combiners within the group are able to provide significant gains. Finally, CI combiners are able to collectively provide a performance improvement over almost all the considered metrics. The sole exception is represented by precision (which is slightly degraded for all the combiners within this group), with a consequent gain reduction of the cor- responding highest F-measure achieved by the CI group. This is more evident when a soft combiner is employed.
(a) Class-conscious (CC) non-trainable combiners (1). (b) Class-conscious (CC) non-trainable combiners (2). Table 4: Performance (%) of different classes of soft combiners, considering Android (iOS) traffic.
(a) Class-conscious (CC) non-trainable combiners (1). (b) Class-conscious (CC) non-trainable combiners (2). Table 4: Performance (%) of different classes of soft combiners, considering Android (iOS) traffic.
Table 5: Maximum Improvement Over Best Classifier (MIOBC) of the F-measure (%) for each class of hard and soft combiners, considering Android (iOS) traffic. explained as CC non-trainable combiners are more prone to be biased from wrong classifiers in the pool, due to the lack of high-level (validation-based) training. Nevertheless, the latter sub-group possesses an intrinsic robustness (due to their pecu- iar combination functions) to having outliers in the pool. On the other hand, by observing performance of CC trainable com- biners (cf. Tab.|7c), it is apparent that KL weights benefits from a judicious use of the subset. This allows reducing the number of parameters to be trained, especially those related to weak classifiers, and thus avoid slight over-fitting. A different trend is instead observed for K weights, being similar to that observed for CC non-trainable combiners. The reason is that the linear (separating) vector employed is based on the assump- tion that each soft-output well-matches (i.e. except for some estimation noise) the actual one [40]. Therefore, this approach is potentially sensitive to erroneous (i.e. providing incoherent soft-outputs) base classifiers. A somewhat similar behavior as BKS is observed for Fuzzy Integral, which does not benefit from subset selection. This may be attributed to the fact that the proposed fuzzy-based fusion design is resistant to classifiers’ uncertainty. A less evident trend can be drawn for CI combin- ers (cf. Tab.|7d). Nonetheless, it can be concluded how all the proposed approaches achieve the highest F-measure with the group considered by Her_Cos, Lib_NB, and Tay_RF, with the sole exception of DT-FSD in Android traffic, where the smallest group composed by Her_Cos and Tay_RF should be employed to reach the highest performance.
Table 5: Maximum Improvement Over Best Classifier (MIOBC) of the F-measure (%) for each class of hard and soft combiners, considering Android (iOS) traffic. explained as CC non-trainable combiners are more prone to be biased from wrong classifiers in the pool, due to the lack of high-level (validation-based) training. Nevertheless, the latter sub-group possesses an intrinsic robustness (due to their pecu- iar combination functions) to having outliers in the pool. On the other hand, by observing performance of CC trainable com- biners (cf. Tab.|7c), it is apparent that KL weights benefits from a judicious use of the subset. This allows reducing the number of parameters to be trained, especially those related to weak classifiers, and thus avoid slight over-fitting. A different trend is instead observed for K weights, being similar to that observed for CC non-trainable combiners. The reason is that the linear (separating) vector employed is based on the assump- tion that each soft-output well-matches (i.e. except for some estimation noise) the actual one [40]. Therefore, this approach is potentially sensitive to erroneous (i.e. providing incoherent soft-outputs) base classifiers. A somewhat similar behavior as BKS is observed for Fuzzy Integral, which does not benefit from subset selection. This may be attributed to the fact that the proposed fuzzy-based fusion design is resistant to classifiers’ uncertainty. A less evident trend can be drawn for CI combin- ers (cf. Tab.|7d). Nonetheless, it can be concluded how all the proposed approaches achieve the highest F-measure with the group considered by Her_Cos, Lib_NB, and Tay_RF, with the sole exception of DT-FSD in Android traffic, where the smallest group composed by Her_Cos and Tay_RF should be employed to reach the highest performance.
(d) Class-indifferent (CI) combiners: Yecision Templates (DT) and Dempster-Shafer (DS) approaches.
(d) Class-indifferent (CI) combiners: Yecision Templates (DT) and Dempster-Shafer (DS) approaches.
Figure 5: Confusion matrices of the best (a) base classifier, (b) hard combiner, (c) soft combiner (Android dataset). Note that the labels (d) a1 ranked according to decreasing abundance of samples and the logarithmic scale (d) is used to evidence small errors.
Figure 5: Confusion matrices of the best (a) base classifier, (b) hard combiner, (c) soft combiner (Android dataset). Note that the labels (d) a1 ranked according to decreasing abundance of samples and the logarithmic scale (d) is used to evidence small errors.
Table 6: F-measure (%) of hard combiners as function of the pool of selected classifiers considering Android (iOS) traffic. Highlighted values: maximum per pool, maximum per combiner, overall maximum. 7: F-measure (%) of soft combiners as function of the pool of selected classifiers considering Android (iOS) traffic. Highlighted values: maximum per pool, maximum per combiner, overall maximum.
Table 6: F-measure (%) of hard combiners as function of the pool of selected classifiers considering Android (iOS) traffic. Highlighted values: maximum per pool, maximum per combiner, overall maximum. 7: F-measure (%) of soft combiners as function of the pool of selected classifiers considering Android (iOS) traffic. Highlighted values: maximum per pool, maximum per combiner, overall maximum.
(a) Class-conscious (CC) non-trainable combiners (1). (b) Class-conscious (CC) non-trainable combiners (2).
(a) Class-conscious (CC) non-trainable combiners (1). (b) Class-conscious (CC) non-trainable combiners (2).
(c) Class-conscious (CC) trainable combiners.
(c) Class-conscious (CC) trainable combiners.
Table 8: Maximum Improvement Over Best Classifier (MIOBC) of the F-measure (%), as function of the pool of selected classifiers, for each class of hard and soft combiners, considering Android (iOS) traffic.
Table 8: Maximum Improvement Over Best Classifier (MIOBC) of the F-measure (%), as function of the pool of selected classifiers, for each class of hard and soft combiners, considering Android (iOS) traffic.
Related topics:
Mobile appsInformation FusionMulti Sensor Data FusionInternet Traffic Classification
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved