DeEPs: A New Instance-based Discovery and Classification System

Lecture Notes in Computer Science, 1998

This paper introduces a new classification algorithm of the instance-based learning type. Training records are converted into patterns associated with a known class label, and stored permanently into a trie 1 -like tree structure along with other helpful information. Classifying new records is done selecting from the trie two best patterns as solutions hypotheses. Best pattern selection is done using standard distance metrics, a strength function and an exclusive values concept. Classification tests done on several data files have shown very accurate results.

Computational Intelligence, 2000

The basic nearest-neighbor rule generalizes well in many domains but has several shortcomings, including inappropriate distance functions, large storage requirements, slow execution time, sensitivity to noise, and an inability to adjust its decision boundaries after storing the training data. This paper proposes methods for overcoming each of these weaknesses and combines these methods into a comprehensive learning system called the Integrated Decremental Instance-Based Learning Algorithm (IDIBL) that seeks to reduce storage, improve execution speed, and increase generalization accuracy, when compared to the basic nearest neighbor algorithm and other learning models. IDIBL tunes its own parameters using a new measure of fitness that combines confidence and cross-validation (CVC) accuracy in order to avoid discretization problems with more traditional leave-one-out cross-validation (LCV). In our experiments IDIBL achieves higher generalization accuracy than other less comprehensive instance-based learning algorithms, while requiring less than onefourth the storage of the nearest neighbor algorithm and improving execution speed by a corresponding factor. In experiments on 21 datasets, IDIBL also achieves higher generalization accuracy than those reported for 16 major machine learning and neural network models.

Journal of Artificial Intelligence and Data Mining, 2015

In instance-based learning, a training set is given to a classifier for classifying new instances. In practice, not all information in the training set is useful for classifiers. Therefore, it is convenient to discard irrelevant instances from the training set. This process is known as instance reduction, which is an important task for classifiers since through this process the time for classification or training could be reduced. Instance-based learning methods are often confronted with the difficulty of choosing the instances, which must be stored to be used during an actual test. Storing too many instances may result in large memory requirements and slow execution speed. In this paper, first, a Distance-based Decision Surface (DDS) is proposed and is used as a separate surface between the classes, and then an instance reduction method, which is based on the DDS is proposed, namely IRDDS (Instance Reduction based on Distance-based Decision Surface). Using the DDS with Genetic algorithm selects a reference set for classification. IRDDS selects the most representative instances, satisfying both of the following objectives: high accuracy and reduction rates. The performance of IRDDS is evaluated on real world data sets from UCI repository by the 10-fold cross-validation method. The results of the experiments are compared with some state-of-the-art methods, which show the superiority of the proposed method, in terms of both classification accuracy and reduction percentage.

Computer Modeling and Intelligent Systems

The task of reducing marked large size samples for building diagnostic and recognizing models by precedents is considered. The method allowing to reduce essentially the size of a training sample increasing at the same time its efficiency, through removal of irrelevant and redundant instances is proposed. The given method provides an opportunity to estimate each instance of a training sample by synthesis of an ensemble of weak classifiers using a bagging model, and to create a reduced sample of the most significant instances by estimations. Software is developed to implement the proposed method. This software has been experimentally investigated in solving the task of reducing synthetic and real world data. The results of the conducted experiments allow recommending the use of the developed method and its software realization for solving the task in the sphere of technical diagnostics.

Applied Sciences

Instance selection and construction methods were originally designed to improve the performance of the k-nearest neighbors classifier by increasing its speed and improving the classification accuracy. These goals were achieved by eliminating redundant and noisy samples, thus reducing the size of the training set. In this paper, the performance of instance selection methods is investigated in terms of classification accuracy and reduction of training set size. The classification accuracy of the following classifiers is evaluated: decision trees, random forest, Naive Bayes, linear model, support vector machine and k-nearest neighbors. The obtained results indicate that for the most of the classifiers compressing the training set affects prediction performance and only a small group of instance selection methods can be recommended as a general purpose preprocessing step. These are learning vector quantization based algorithms, along with the Drop2 and Drop3. Other methods are less effi...

2021

e use of machine learning techniques to predict material strength is becoming popular. However, not much attention has been paid to instance-based learning (IBL) algorithms. erefore, in order to predict material strength, as the direct method by conducting tests is time-consuming and expensive and experimental errors are inevitable, an indirect method based on elementary instance-based learning algorithm was proposed. e standard k-nearest neighbors (k-NN) with cross-validation were utilized to develop compressive strength prediction models for some concretes and rocks by considering indirect parameters such as physical and mechanical parameters. Results on applying this method to datasets from literature studies show that the values of RMSE for k-NN are modest, indicating adequacy to predict compressive strength with comprehensive range values of predictors. Additionally, the R 2-values of the k-NN models were high. In other words, the models were able to explain the variance in compressive strength for data with a wide range of input values.

International Database Engineered Applications Symposium Conference

Reduction through Homogeneous Clustering (RHC) and its editing variant (ERHC) are effective data reduction techniques for the k-NN classifier. They are based on an iterative k-means clustering task that discovers homogeneous clusters. The centers of the resulting homogeneous clusters constitute the instances of the reduced training set. Although RHC and ERHC are quite fast compared to several well-known data reduction techniques, the iterative execution of k-means clustering renders both of them inappropriate for data reduction tasks that need to be performed quickly, especially, when run over large training datasets. The present paper proposes simple and very fast variations of the algorithms, which are appropriate for such environments. The variations are called RHC2 and ERHC2 and replace the complete execution of k-means clustering with a fast task that assigns instances to the class centers. The experimental study based on fourteen datasets, and, the corresponding statistical tests, show that the proposed RHC2 and ERHC2 variations are very fast and, at the cost of a small penalty on classification accuracy, they achieve higher reduction rates than their predecessors and other two well-known data reduction techniques. They are good candidates when fast reduction on large datasets is required. CCS CONCEPTS • Information systems → Nearest-neighbor search; • Computing methodologies → Instance-based learning.

Two disadvantages of the standard nearest neighbor algorithm are 1) it must store all the instances of the training set, thus creating a large memory footprint and 2) it must search all the instances of the training set to predict the classification of a new query point, thus it is slow at run time. Much work has been done to remedy these shortcomings. This paper presents a new algorithm WITS (Weighted-Instance Typicality Search) and a modified version, Clustered-WITS (C-WITS), designed to address these issues. Data reduction algorithms address both issues by storing and using only a portion of the available instances. WITS is an incremental data reduction algorithm with O(n 2 ) complexity, where n is the training set size. WITS uses the concept of Typicality in conjunction with Instance-Weighting to produce minimal nearest neighbor solutions. WITS and C-WITS are compared to three other state of the art data reduction algorithms on ten real-world datasets. WITS achieved the highest average accuracy, showed fewer catastrophic failures, and stored an average of 71% fewer instances than DROP-5, the next most competitive algorithm in terms of accuracy and catastrophic failures. The C-WITS algorithm provides a user-defined parameter that gives the user control over the training-time vs. accuracy balance. This modification makes C-WITS more suitable for large problems, the very problems data reductions algorithms are designed for. On two large problems (10,992 and 20,000 instances), C-WITS stores only a small fraction of the instances (0.88% and 1.95% of the training data) while maintaining generalization accuracies comparable to the best accuracies reported for these problems.

Knowledge-Based Systems, 2014

Instance selection is one of the most important preprocessing steps in many machine learning tasks. Due to the increasing size of the problems, removing useless, erroneous or noisy instances is frequently an initial step that is performed before other data mining algorithms are applied. Instance selection as part of this data reduction task is one of the most relevant problems in current data mining research. Over the past decades, many different instance selection algorithms have been proposed, each with its own strengths and weaknesses. However, as in the case of classification, it is unlikely that a single instance selection algorithm would be able to achieve good results across many different datasets and application fields. In classification, one of the most successful ways of consistently improving the performance of a single learner is the construction of ensembles using boosting methods. In this paper, we propose a novel approach for instance selection based on boosting instance selection algorithms in the same way boosting is applied to classification. The proposed approach opens a new field of research in which to apply the many techniques developed for boosting classifiers, for instance selection and other data reduction techniques such as feature selection and simultaneous instance and feature selection. Using 60 datasets for balanced problems and 45 datasets for class-imbalanced problems, the experiments reported show a clear improvement in several state-of-the-art instance selection algorithms using the proposed methodology.