CLASSIFICATION USING INSTANCE-BASED LEARNING PATTERNS

2021

e use of machine learning techniques to predict material strength is becoming popular. However, not much attention has been paid to instance-based learning (IBL) algorithms. erefore, in order to predict material strength, as the direct method by conducting tests is time-consuming and expensive and experimental errors are inevitable, an indirect method based on elementary instance-based learning algorithm was proposed. e standard k-nearest neighbors (k-NN) with cross-validation were utilized to develop compressive strength prediction models for some concretes and rocks by considering indirect parameters such as physical and mechanical parameters. Results on applying this method to datasets from literature studies show that the values of RMSE for k-NN are modest, indicating adequacy to predict compressive strength with comprehensive range values of predictors. Additionally, the R 2-values of the k-NN models were high. In other words, the models were able to explain the variance in compressive strength for data with a wide range of input values.

Distance is used as a measurement in most of instance-based classi cation systems. Rather than using distance, we make use of the frequency of an instance's subsets and the frequency-changing rate of the subsets among training classes to perform both knowledge discovery and classi cation tasks. We name the system DeEPs. Whenever an instance is considered, DeEPs can e ciently discover those patterns contained in the instance which sharply di erentiate the training classes from one to another. If the instance is unseen previously, DeEPs can predict a class label for it by compactly summarizing the frequencies of the discovered patterns based on a view to collectively maximizing the discriminating power of the patterns. According to some interestingness measures, top-ranked patterns by DeEPs can help users conveniently assess the importance of the discovered patterns. Many experimental results are used to evaluate the system, showing that the patterns are comprehensible and that DeEPs is accurate and scalable. Based on the results on 40 benchmark data sets, DeEPs is more accurate than k-nearest neighbor and C5.0.

Applied Sciences

Instance selection and construction methods were originally designed to improve the performance of the k-nearest neighbors classifier by increasing its speed and improving the classification accuracy. These goals were achieved by eliminating redundant and noisy samples, thus reducing the size of the training set. In this paper, the performance of instance selection methods is investigated in terms of classification accuracy and reduction of training set size. The classification accuracy of the following classifiers is evaluated: decision trees, random forest, Naive Bayes, linear model, support vector machine and k-nearest neighbors. The obtained results indicate that for the most of the classifiers compressing the training set affects prediction performance and only a small group of instance selection methods can be recommended as a general purpose preprocessing step. These are learning vector quantization based algorithms, along with the Drop2 and Drop3. Other methods are less effi...

Instance-based learning algorithms are often faced with the problem of deciding which instances to store for use during generalization. Storing too many instances can result in large memory requirements and slow execution speed, and can cause an oversensitivity to noise. This paper has two main purposes. First, it provides a survey of existing algorithms used to reduce storage requirements in instance-based learning algorithms and other exemplar-based algorithms. Second, it proposes six additional reduction algorithms called DROP1{DROP5 and DEL (three of which were rst described in Wilson & Martinez, 1997c, as RT1{RT3) that can be used to prune instances from the concept description. These algorithms and 10 algorithms from the survey are compared on 31 classi cation tasks. Of those algorithms that provide substantial storage reduction, the DROP algorithms have the highest average generalization accuracy in these experiments, especially in the presence of noise.

Computer Modeling and Intelligent Systems

The task of reducing marked large size samples for building diagnostic and recognizing models by precedents is considered. The method allowing to reduce essentially the size of a training sample increasing at the same time its efficiency, through removal of irrelevant and redundant instances is proposed. The given method provides an opportunity to estimate each instance of a training sample by synthesis of an ensemble of weak classifiers using a bagging model, and to create a reduced sample of the most significant instances by estimations. Software is developed to implement the proposed method. This software has been experimentally investigated in solving the task of reducing synthetic and real world data. The results of the conducted experiments allow recommending the use of the developed method and its software realization for solving the task in the sphere of technical diagnostics.

Swarm and Evolutionary Computation, 2018

Instance-based learning methods, such as the k-nearest neighbor rule, are among the top-performing methods in any classification task. Despite their simplicity, they achieve comparable performance to much more complex methods. However, one of their problems is the necessity to store all of the training instances in memory. For large datasets, this might also affect the speed of the testing process. As a solution to this problem, instance selection methods, which remove redundant and noisy instances, have been proposed. However, a general side effect of these methods is that they produce a significant reduction in the accuracy of the instance-based learner. The classification performance is significantly worse compared to the application of the k-nearest neighbor rule using all instances. In this paper, we propose an evolutionary instance selection algorithm that combines three strategies to avoid this negative side effect. First, it uses the framework of a CHC genetic algorithm, as it has been proved to be the best-performing method for this task. Second, it incorporates the possibility of selecting each instance more than once. This has also been proven useful in previous works. Finally, it uses a local value for k that depends on the nearest neighbor of every test instance. These three combined strategies are able to achieve better reduction than previous approaches while maintaining the same classification performance as the k-nearest neighbor rule.

Pattern Recognition, 2019

Instance selection is one of the most important preprocessing steps in many machine learning tasks. Due to huge data size that is common in many current problems, removing redundant, useless, erroneous or noisy instances is a frequent initial step that is performed before other data mining algorithms are applied. Instance selection as part of this data reduction task is a relevant problem in current data mining research. Many instance selection methods hypothesize a certain way of characterizing the most important instances and then implement an algorithm to keep those important instances. The problem with these methods is that their success depends on the data fulfilling the underlying hypothesis. Other methods just add or delete instances considering only their effect on the accuracy of the nearest neighbor rule. In this paper, we present a new method of this second kind that uses boosting to obtain a subset of instances that is able to improve the classification accuracy of the whole dataset with a significant reduction. The instances are incrementally added by selecting those that maximize the accuracy of the subset using the weighting of instances from the construction of ensembles of classifiers and the stepwise addition of new instances. The method is compared using a large set of 205 different datasets with standard methods and shows the best overall performance.

Journal of Intelligent Systems, 2012

Network data analysis helps in capturing node usage behavior. Existing algorithms use reduced feature set to manage high runtime complexity. Ignoring features may increase classification errors. This paper presents a model, allowing classification of network traffic, while considering all the relevant features. Learning phase partitions training sample on values of the respective features. This creates equivalence classes related to m features. During classification, each feature value of the test instance results in picking one set from equivalence class generated during learning. Algorithm captures new behavior in semi-supervised incremental learning mode. For problems having m features and n training samples the model has incremental learning complexity of O.m/ and average classification complexity is of the order O.m.log 2 n/ 2 /.

2009

Abstract Instance-based classifiers that compute similarity between instances suffer from the presence of noise in the training set and from over-fitting. In this paper we propose a new type of distance-based classifier that instead of computing distances between instances computes the distance between each test instance and the classes. Both are represented by patterns in the space of the frequent itemsets. We ranked the itemsets by metrics of itemset significance.

One of the most important challenges in developing problem solving methods is to combine and synergistically utilize general and specific knowledge. This paper presents one possible way of performing this integration that might be generally described as follows: "To solve a problem, first try to use the conventional rulebased approach. If it does not work, try to find a similar problem you have solved in the past and adapt the old solution to the new situation". We applied this heuristic for a classification task. The background concepts of this heuristic are standard cases (the source of data for the rules) and exceptional cases (representative of the specific knowledge). The presented empirical study has not shown this attempt to be successful in accuracy when it is compare to its parents methods: instance-based and rulebased approach, but a careful policy in the distribution of standard and exceptional cases might provide a competitive classifier in terms of accuracy and comprehensibility.