Academia.eduAcademia.edu

Figure 13 – uploaded by Raj Jain

See full PDFdownloadDownload figure
For Level-2 detection, we have chosen One-R as the baseline algorithm. One-R is a simple but accurate classification algorithm, which generates one rule for each predictor and then selects the one with the smallest error. The accuracy of our framework is 13.03% better for ‘impending’ faults and 5.97% better for ‘manifested’ faults, which is a significant improvement (Fig. 13.) Fig. 13. Detection Level 2 effectiveness compared to baseline

Figure 13 For Level-2 detection, we have chosen One-R as the baseline algorithm. One-R is a simple but accurate classification algorithm, which generates one rule for each predictor and then selects the one with the smallest error. The accuracy of our framework is 13.03% better for ‘impending’ faults and 5.97% better for ‘manifested’ faults, which is a significant improvement (Fig. 13.) Fig. 13. Detection Level 2 effectiveness compared to baseline

Related Figures (35)

Fig. 1. A carrier’s broadband service network
Fig. 1. A carrier’s broadband service network
The broadband services, like other carrier services, are currently provided through networks constructed from physical appliances like routers, aggregation switches and Digital Subscriber Line Access Multiplexers (DSLAMs) from various Original Equipment Manufacturers (OEMs). In discussing the
The broadband services, like other carrier services, are currently provided through networks constructed from physical appliances like routers, aggregation switches and Digital Subscriber Line Access Multiplexers (DSLAMs) from various Original Equipment Manufacturers (OEMs). In discussing the
Fig. 3. Virtual broadband service The figure also shows that the carrier has retained DSLAMs as Physical Network Function (PNF) from their legacy network, as these functions might not have reached their end-of-life. VNFs of a service may belong to different vendors, owned by different operators, managed by different platforms and even unaware of each other. In such a case, the service is a multi- domain VNS [16].
Fig. 3. Virtual broadband service The figure also shows that the carrier has retained DSLAMs as Physical Network Function (PNF) from their legacy network, as these functions might not have reached their end-of-life. VNFs of a service may belong to different vendors, owned by different operators, managed by different platforms and even unaware of each other. In such a case, the service is a multi- domain VNS [16].
Fig. 4. SFC created on multiple clouds Fig. 4 shows an S VNF2-VNF3-VNF4-V VNF1-VNF2-VNF3-V FC with two paths, i.e., PNFI-VNF1- NF6 for the Internet access and PNF1- NF5-VNF8 for content services. As can be seen in this figure, there are multiple paths through meshed core routers through which traffic can be routed if the selected link fails or if there is congestion on the selected link. VNFs of an SFC are connected in the same manner as the physical
Fig. 4. SFC created on multiple clouds Fig. 4 shows an S VNF2-VNF3-VNF4-V VNF1-VNF2-VNF3-V FC with two paths, i.e., PNFI-VNF1- NF6 for the Internet access and PNF1- NF5-VNF8 for content services. As can be seen in this figure, there are multiple paths through meshed core routers through which traffic can be routed if the selected link fails or if there is congestion on the selected link. VNFs of an SFC are connected in the same manner as the physical
A VNF is the virtual counterpart of a network appliance or a middlebox im off-the-shelf plemented by running software over commercial general purpose servers. The software to implement a network function may run over a general-purpose physical mac physical mac also be instan hine or over the virtual machine(s) created on hines using virtualization software. VNFs can tiated on virtual resources obtained from one or more cloud service providers. Each VNF has a well-defined functional be havior and interfaces for interconnection with other VNFs or PNFs. Fig. 5 shows a VNF with its Element Management System (EMS) and interfaces to the rest of the network [19] 20). Some examples of pre-programmed VNFs are given in Table 1.
A VNF is the virtual counterpart of a network appliance or a middlebox im off-the-shelf plemented by running software over commercial general purpose servers. The software to implement a network function may run over a general-purpose physical mac physical mac also be instan hine or over the virtual machine(s) created on hines using virtualization software. VNFs can tiated on virtual resources obtained from one or more cloud service providers. Each VNF has a well-defined functional be havior and interfaces for interconnection with other VNFs or PNFs. Fig. 5 shows a VNF with its Element Management System (EMS) and interfaces to the rest of the network [19] 20). Some examples of pre-programmed VNFs are given in Table 1.
Fig. 6. Network function virtualization infrastructure Protocols for routing of traffic, through an SFC, are being worked upon by the standards organizations. For example, two protocols - Segment Routing (SR) and the Network Service Header (NSH) - are under development in the IETF [84]. SR is a modified version of source routing. In SR the IPv6 header is extended to include the Segment Routing Header (SRH), which decides the path of the traffic packets. A segment is a path through a carrier network. The internals of the segment may not be exposed to the users. For example, it may be a Multi-Protocol Label Switching (MPLS) tunnel or may be a sequence of IP routers. Each segment has an ID and may contain information about the treatment of the traffic on that segment. A Software Defined Networking (SDN) controller may utilize the Path Computation Element Protocol (PCEP) to find the most appropriate segments and instruct the classifier to direct the traffic flow accordingly. NSH, on the other hand, can work with IPv4, IPv6, and Ethernet. NSH is an 8-byte header followed by a number of optional variable length context headers containing some metadata to be used by NSH-aware devices. Implementation of service function chains with NSH capabilities requires NSH-aware virtual switches and a central controller.
Fig. 6. Network function virtualization infrastructure Protocols for routing of traffic, through an SFC, are being worked upon by the standards organizations. For example, two protocols - Segment Routing (SR) and the Network Service Header (NSH) - are under development in the IETF [84]. SR is a modified version of source routing. In SR the IPv6 header is extended to include the Segment Routing Header (SRH), which decides the path of the traffic packets. A segment is a path through a carrier network. The internals of the segment may not be exposed to the users. For example, it may be a Multi-Protocol Label Switching (MPLS) tunnel or may be a sequence of IP routers. Each segment has an ID and may contain information about the treatment of the traffic on that segment. A Software Defined Networking (SDN) controller may utilize the Path Computation Element Protocol (PCEP) to find the most appropriate segments and instruct the classifier to direct the traffic flow accordingly. NSH, on the other hand, can work with IPv4, IPv6, and Ethernet. NSH is an 8-byte header followed by a number of optional variable length context headers containing some metadata to be used by NSH-aware devices. Implementation of service function chains with NSH capabilities requires NSH-aware virtual switches and a central controller.
Fig. 7. Orchestration and management of VNS Fig. /. Orchestration and management of VNS In view of what has been said above, the fault management function becomes a shared responsibility. The relative distribution of responsibilities among various platforms and
Fig. 7. Orchestration and management of VNS Fig. /. Orchestration and management of VNS In view of what has been said above, the fault management function becomes a shared responsibility. The relative distribution of responsibilities among various platforms and
COMPARISON OF SOME COMPETITIVE NETWORK SERVICE ORCHESTRATION SOLUTIONS FROM FP PERSPECTIVE TABLE 3
COMPARISON OF SOME COMPETITIVE NETWORK SERVICE ORCHESTRATION SOLUTIONS FROM FP PERSPECTIVE TABLE 3
SHARED FP RESPONSIBILITIES OF DIFFERENT MANAGEMENT ENTITIES TABLE 4
SHARED FP RESPONSIBILITIES OF DIFFERENT MANAGEMENT ENTITIES TABLE 4
TABLE 5 We have introduced markers by an operational network and before as indicators produced measurements taken by the operations staff. There are a large number of markers that are directly or indirectly related to t he occurrence of an FP issue. These markers become important features in our datasets. Events, that produce these markers, relate to communication, QoS, processing, equipment, and environment. Of course, not only each FP issue would usual also many of the markers would of issue. Also, at any given time y have multiple markers, but appear in more than one type the markers produced may be a result of more than one FP issue. Thus, there is a complex relationship between the markers and the FP issues. This would usually mean that when using machine learning for fault detection and localization, feature engineering, i.e., selection of appropriate markers would be required to get better results. However, deep-learning models , are able to extract relevant features automatically, without human intervention. Some of the markers related to mobile, fixed and broadband networks are given in Table 5.
TABLE 5 We have introduced markers by an operational network and before as indicators produced measurements taken by the operations staff. There are a large number of markers that are directly or indirectly related to t he occurrence of an FP issue. These markers become important features in our datasets. Events, that produce these markers, relate to communication, QoS, processing, equipment, and environment. Of course, not only each FP issue would usual also many of the markers would of issue. Also, at any given time y have multiple markers, but appear in more than one type the markers produced may be a result of more than one FP issue. Thus, there is a complex relationship between the markers and the FP issues. This would usually mean that when using machine learning for fault detection and localization, feature engineering, i.e., selection of appropriate markers would be required to get better results. However, deep-learning models , are able to extract relevant features automatically, without human intervention. Some of the markers related to mobile, fixed and broadband networks are given in Table 5.
The metrics used by carriers to measure the health of the network provide important information about the FP problems at the macro level. Use of these as features in the dataset would help learning algorithms to narrow down the scope of localization effort. According to ITU Recommendation regarding the QoS criteria and parameters, a number of basic aspects have to be considered while identifying measurable metrics of service availability [59]. ETSI documents on service availability [56] and on service quality [60] mention metrics that need to be collected and analyzed. The ETSI group specification on service quality metrics [61] recognizes that it is important to have an objective and quantitative metrics to assist in identifying problems when they arise and provide good service to the consumers. Examples of metrics, and their realistic values (where applicable), from an actual network [62] are given in Table 6. We will see how these markers and metrics are used in our framework in later sections.
The metrics used by carriers to measure the health of the network provide important information about the FP problems at the macro level. Use of these as features in the dataset would help learning algorithms to narrow down the scope of localization effort. According to ITU Recommendation regarding the QoS criteria and parameters, a number of basic aspects have to be considered while identifying measurable metrics of service availability [59]. ETSI documents on service availability [56] and on service quality [60] mention metrics that need to be collected and analyzed. The ETSI group specification on service quality metrics [61] recognizes that it is important to have an objective and quantitative metrics to assist in identifying problems when they arise and provide good service to the consumers. Examples of metrics, and their realistic values (where applicable), from an actual network [62] are given in Table 6. We will see how these markers and metrics are used in our framework in later sections.
Fig. 6. [he FP management framework input with good accuracy. Good reconstruction performance helps in achieving good prediction. For prediction of fault classes or severity of impending faults, a layer of Softmax classifier replaces the decoder layers with h” forming the
Fig. 6. [he FP management framework input with good accuracy. Good reconstruction performance helps in achieving good prediction. For prediction of fault classes or severity of impending faults, a layer of Softmax classifier replaces the decoder layers with h” forming the
Fig. 9 The detection subsystem We propose learning models that have predictive and deductive properties to meet the FP requirements of virtua network services. All the markers available from the management platforms, i.e., the runtime monitoring and measurements, alarms, notifications and warnings configuration changes, and environmental factors are used along with machine learning models trained with historica data to draw inferences about the manifested performance and fault issues. Additionally, the capability of deep learning to map the intricate relationship among the features has been used to predict the impending faults. The framework shown in Fig. 8 consists of three main sub-systems: Data pre-processing, Detection and Localization. Data pre-processing involves collation and normalization of the dataset to remove biases. The pre-processing policy may also involve the reduction of features based on some criterion like correlation with the labels. In the training mode, the available dataset is split into training and test datasets, which are used to train and test all the models. During operation, the marker data is run through the framework to detect and localize problems.
Fig. 9 The detection subsystem We propose learning models that have predictive and deductive properties to meet the FP requirements of virtua network services. All the markers available from the management platforms, i.e., the runtime monitoring and measurements, alarms, notifications and warnings configuration changes, and environmental factors are used along with machine learning models trained with historica data to draw inferences about the manifested performance and fault issues. Additionally, the capability of deep learning to map the intricate relationship among the features has been used to predict the impending faults. The framework shown in Fig. 8 consists of three main sub-systems: Data pre-processing, Detection and Localization. Data pre-processing involves collation and normalization of the dataset to remove biases. The pre-processing policy may also involve the reduction of features based on some criterion like correlation with the labels. In the training mode, the available dataset is split into training and test datasets, which are used to train and test all the models. During operation, the marker data is run through the framework to detect and localize problems.
redundancies, as a good amount of similar information may be available from OSS and MANO. However, making use of redundant data makes up for the gaps in communication among various management platforms.
redundancies, as a good amount of similar information may be available from OSS and MANO. However, making use of redundant data makes up for the gaps in communication among various management platforms.
Thus, while manifested faults are manageable with shallow models, impending faults have been tackled with deep
Thus, while manifested faults are manageable with shallow models, impending faults have been tackled with deep
networks can be tuned well for sparse data by using parameters like sparsity regularization and sparsity proportion as discussed in the evaluation section.
networks can be tuned well for sparse data by using parameters like sparsity regularization and sparsity proportion as discussed in the evaluation section.
Fig. 12. The stacked encoder used for prediction 7. Evaluation of the model
Fig. 12. The stacked encoder used for prediction 7. Evaluation of the model
The Telstra log_feature sub-dataset contains 58,672 examples, with events displaying the presence of different features. The event_type sub-dataset has 31,170 examples, the resource_type sub-dataset has 21,076 and severity_type sub- dataset has 18,552 examples. The test and the training files have 11,171 and 7381 records respectively. They have not been split from a common dataset so the standard 80:20 or a similar ratio is not maintained. A dataset prepared by the consolidation of all sub-datasets has more than 800 features as shown in Table 10. Each fault (with a unique id) is associated with a location, up to 6 features and corresponding volumes, up to three affected resources, up to 5 events, and up to 5 severity types indicating the intensity of the warning and Tr fault’ and 1 indicating ‘fault.’ The detection classification of ‘fault’/‘no-fault’? was implemented with a number of supervised learning techniques of which SVM, ADT, and RF have been
The Telstra log_feature sub-dataset contains 58,672 examples, with events displaying the presence of different features. The event_type sub-dataset has 31,170 examples, the resource_type sub-dataset has 21,076 and severity_type sub- dataset has 18,552 examples. The test and the training files have 11,171 and 7381 records respectively. They have not been split from a common dataset so the standard 80:20 or a similar ratio is not maintained. A dataset prepared by the consolidation of all sub-datasets has more than 800 features as shown in Table 10. Each fault (with a unique id) is associated with a location, up to 6 features and corresponding volumes, up to three affected resources, up to 5 events, and up to 5 severity types indicating the intensity of the warning and Tr fault’ and 1 indicating ‘fault.’ The detection classification of ‘fault’/‘no-fault’? was implemented with a number of supervised learning techniques of which SVM, ADT, and RF have been
LIST OF FEATURES FROM NETWORK FAULT DATASET A part of the consolidated Telstra dataset is shown in Table 11. Only feature] (out of the complete set of features from feature! to feature386) is s hown for compactness. As part of preprocessing of the dataset, selection of features was carried out based on the degree of correlation of each feature with the labels using the Weka tool study, a correlation thresho accuracy. [78]. With the dataset used in this d of 23% was found to improve
LIST OF FEATURES FROM NETWORK FAULT DATASET A part of the consolidated Telstra dataset is shown in Table 11. Only feature] (out of the complete set of features from feature! to feature386) is s hown for compactness. As part of preprocessing of the dataset, selection of features was carried out based on the degree of correlation of each feature with the labels using the Weka tool study, a correlation thresho accuracy. [78]. With the dataset used in this d of 23% was found to improve
With this dataset, SVM, on the whole, performs better ADT and RF giving = 95.4% accuracy. Considering definitions in Table 13, the true positive (TP) rate for ‘fault’ cases were the highest for SVM showing that these were correctly classified as ‘fault’ cases. Considering the nature of the dataset, this result indicates a good result. There were no fau fau pos hig fau per ts and system said no faults in 2.4% cases. The formance and fault issues. SVM and RF also gave classified by them. than the ts and system said fault in 5.7% cases, while there were false itive and negative rates were the lowest in SVM and the hest in Random Forest. A desirable outcome is that besides classifying faults and faults and no faults as no faults with accuracy, it classifies a very low percentage of faults as ts, thus, helping to do what is intended to do — detect high no- high precision indicating that ‘no-fault’ cases were correctly shown in Table 12. On the basis of accuracy, SVM and ADT perform comparatively better than RF. In each case, 10% cross- validation was used.
With this dataset, SVM, on the whole, performs better ADT and RF giving = 95.4% accuracy. Considering definitions in Table 13, the true positive (TP) rate for ‘fault’ cases were the highest for SVM showing that these were correctly classified as ‘fault’ cases. Considering the nature of the dataset, this result indicates a good result. There were no fau fau pos hig fau per ts and system said no faults in 2.4% cases. The formance and fault issues. SVM and RF also gave classified by them. than the ts and system said fault in 5.7% cases, while there were false itive and negative rates were the lowest in SVM and the hest in Random Forest. A desirable outcome is that besides classifying faults and faults and no faults as no faults with accuracy, it classifies a very low percentage of faults as ts, thus, helping to do what is intended to do — detect high no- high precision indicating that ‘no-fault’ cases were correctly shown in Table 12. On the basis of accuracy, SVM and ADT perform comparatively better than RF. In each case, 10% cross- validation was used.
To get a sense of the performance of our detection model, using SVM with RBF Kernel, we compared the results with baseline results obtained by Zero-R model. The Zero-R model predicts the majority class. Running on our datasets, the baseline result was about 63%, which indicates that our chosen model gives a substantial improvement over the naive baseline.
To get a sense of the performance of our detection model, using SVM with RBF Kernel, we compared the results with baseline results obtained by Zero-R model. The Zero-R model predicts the majority class. Running on our datasets, the baseline result was about 63%, which indicates that our chosen model gives a substantial improvement over the naive baseline.
Fig. 15. Localization Layer 2 effectiveness compared to baseline amples. Fig. 15 gives the graphical comparison of Level 2 performance of the implemented model (MCML: Multi-class, Multi-layer, in our case SMO) and the baseline. It is seen that the implemented model gives a higher percentage of correctly classified and lower percentage of wrongly classified
Fig. 15. Localization Layer 2 effectiveness compared to baseline amples. Fig. 15 gives the graphical comparison of Level 2 performance of the implemented model (MCML: Multi-class, Multi-layer, in our case SMO) and the baseline. It is seen that the implemented model gives a higher percentage of correctly classified and lower percentage of wrongly classified
Fig. 14 gives a comparison of the performance of our Multi- Class Multi-Layer (MCML) model with the baseline. It can be seen that the accuracy of the classification of MCML is 97.03% against the baseline accuracy of 86.14%. A useful metric for comparison of classifiers is Precision-Recall Area (PRC Area), which gives the tradeoff between precision and recall. A high value indicates high precision (i.e., low false positives) and high recall (i.e., low false negatives). We can see that MCML Level 1 gives a high PRC Area of 0.967 compared to 0.806 of the baseline.
Fig. 14 gives a comparison of the performance of our Multi- Class Multi-Layer (MCML) model with the baseline. It can be seen that the accuracy of the classification of MCML is 97.03% against the baseline accuracy of 86.14%. A useful metric for comparison of classifiers is Precision-Recall Area (PRC Area), which gives the tradeoff between precision and recall. A high value indicates high precision (i.e., low false positives) and high recall (i.e., low false negatives). We can see that MCML Level 1 gives a high PRC Area of 0.967 compared to 0.806 of the baseline.
LOCALIZATION LAYER 2 BASELINE PERFORMANCE
LOCALIZATION LAYER 2 BASELINE PERFORMANCE
TABLE 17 A seen from Table 18, when compared with the baseline algorithm result, we see that multi-class classification with SMO and OvO has a much superior performance, indicating the efficacy of the model. The localization accuracy of the model is 96.04% compared to 90.1% of baseline. The PRC Area of the MCML Level 2 classification is 0.955 against 0.846 of the baseline.
TABLE 17 A seen from Table 18, when compared with the baseline algorithm result, we see that multi-class classification with SMO and OvO has a much superior performance, indicating the efficacy of the model. The localization accuracy of the model is 96.04% compared to 90.1% of baseline. The PRC Area of the MCML Level 2 classification is 0.955 against 0.846 of the baseline.
To make the deep learning model predict with high accuracy, the first step is to train the stacked autoencoders such that the output is as close a replica of the input as possible. To achieve optimum performance, the stacked autoencoder parameters like the number of hidden layers, the size of the layers, sparsity regulation (SR) and sparsity proportion need to be judiciously arrived at. An example of comparative reconstruction performance is given in Figures 16(a) through 16(d). It is seen that the model with 3 hidden layers of 200/150/100 neurons, respectively, converges quite fast to a low mean-square error. Reconstruction accuracy is important as it affects the prediction based on the trained encoders, which the model is eventually used for [22]. 8. Summary Fig. 16. Mean square error for reconstruction of the input
To make the deep learning model predict with high accuracy, the first step is to train the stacked autoencoders such that the output is as close a replica of the input as possible. To achieve optimum performance, the stacked autoencoder parameters like the number of hidden layers, the size of the layers, sparsity regulation (SR) and sparsity proportion need to be judiciously arrived at. An example of comparative reconstruction performance is given in Figures 16(a) through 16(d). It is seen that the model with 3 hidden layers of 200/150/100 neurons, respectively, converges quite fast to a low mean-square error. Reconstruction accuracy is important as it affects the prediction based on the trained encoders, which the model is eventually used for [22]. 8. Summary Fig. 16. Mean square error for reconstruction of the input
Fig. 18. Localization of impending fault stacked autoencoder and SVM (baseline) We baselined the above results with those obtained with a shallow model, viz., SVM with RBF kernel which worked very well for detection and could only obtain 73.1% accuracy in localizing impending faults. A comparison between SSAE and SVM models is shown in Fig. 18. The deep structure thus provided a substantial improvement in terms of accuracy of prediction of the severity level of the impending faults.
Fig. 18. Localization of impending fault stacked autoencoder and SVM (baseline) We baselined the above results with those obtained with a shallow model, viz., SVM with RBF kernel which worked very well for detection and could only obtain 73.1% accuracy in localizing impending faults. A comparison between SSAE and SVM models is shown in Fig. 18. The deep structure thus provided a substantial improvement in terms of accuracy of prediction of the severity level of the impending faults.
Fine tuning of the model was done using backpropagation. The accuracy ranges between 72 and 85% with the abridged dataset (~1000 examples) and ~92% with the enhanced dataset (~5000 examples). Experiments were carried out for SR = | and SP=04.
Fine tuning of the model was done using backpropagation. The accuracy ranges between 72 and 85% with the abridged dataset (~1000 examples) and ~92% with the enhanced dataset (~5000 examples). Experiments were carried out for SR = | and SP=04.
Related topics:
EngineeringTechnologyComputer Networks
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved