K-winner machines for pattern classification

Emerging Technologies in Data Mining and Information Security

This study aims to implement the following four advanced pattern recognition algorithms, such as "optimal Bayesian classifier," "anti-Bayesian classifier," "decision trees (DTs)," and "dependence trees (DepTs)" on both artificial and real datasets for multi-class classification. Then, we calculated the performance of individual algorithms on both real and artificial data for comparison. In Sect. 1, a brief introduction is given about the study. In the second section, the different types of datasets used in this study are discussed. In the third section, we compared the classification accuracies of Bayesian and anti-Bayesian methods for both the artificial and real-life datasets. In the fourth section, a comparison between the classification accuracy of DT and DepT classification methods for both the artificial and real-life datasets is discussed. In the fifth section, a comparison between the classification accuracy of the four algorithms, such as (a) Bayes, (b) anti-Bayes, (c) DTs, and (d) DepTs for both the artificial and real datasets is explained. We used 5fold cross-validation to determine the classification accuracy of individual, machine learning-based, advanced pattern recognition (PR) models.

International Journal of Artificial Intelligence and Soft Computing, 2009

Learning Vector Quantisation (LVQ) is a method of applying the Vector Quantisation (VQ) to generate references for Nearest Neighbour (NN) classification. Though successful in many occasions, LVQ suffers from several shortcomings, especially the reference vectors are prone to diverge. In this paper, we propose a Classified Vector Quantisation (CVQ) to establish VQ for classification. By CVQ, each data category is represented by its own codebook, which can be implemented by some learning algorithms. In classification process, each codebook offers a generalised NN. The examples of handwritten digit recognition and offline signature verification are used to demonstrate the efficiency of the proposed scheme.

2002

In this paper, we propose a method for multi-class pattern classification by combined use of Multinomial Logit Model (MLM) and K-nearest neighbor rule (K-NN). Multinomial Logit Model (MLM) is one of the neural network models for multi-class pattern classification, and is supposed to be equal or better in class&cation performance than linear classification methods. K-NN is a simple but powerful non-parametric classification tool whose error probability does not exceed double of b y e s error. However, it is also known that such high performance of K-NN is not always expected if number of dimension of input feature vector space is large. Therefore, first we train MLM using the training vectors, and then apply K-NN to the output of the MLM. By this, since K-NN is applied to the compressed low dimension vectors, it is expected not only to bring out natural performance of K-NN but also to shorten computation time.

Pattern Recognition, 2007

Prototype classifiers are a type of pattern classifiers, whereby a number of prototypes are designed for each class so as they act as representatives of the patterns of the class. Prototype classifiers are considered among the simplest and best performers in classification problems. However, they need careful positioning of prototypes to capture the distribution of each class region and/or to define the class boundaries. Standard methods, such as learning vector quantization (LVQ), are sensitive to the initial choice of the number and the locations of the prototypes and the learning rate. In this article, a new prototype classification method is proposed, namely self-generating prototypes (SGP). The main advantage of this method is that both the number of prototypes and their locations are learned from the training set without much human intervention. The proposed method is compared with other prototype classifiers such as LVQ, self-generating neural tree (SGNT) and K-nearest neighbor (K-NN) as well as Gaussian mixture model (GMM) classifiers. In our experiments, SGP achieved the best performance in many measures of performance, such as training speed, and test or classification speed. Concerning number of prototypes, and test classification accuracy, it was considerably better than the other methods, but about equal on average to the GMM classifiers. We also implemented the SGP method on the well-known STATLOG benchmark, and it beat all other 21 methods (prototype methods and non-prototype methods) in classification accuracy.

Neurocomputing, 2003

We propose an algorithm for training multi layer preceptrons (MLP) for classiÿcation problems, that we named hidden layer learning vector quantization. It consists of applying learning vector quantization to the last hidden layer of a MLP and it gave very successful results on problems containing a large number of correlated inputs. It was applied with excellent results on classiÿcation of Rutherford backscattering spectra and to a benchmark problem of image recognition.

1994

Abstract This paper compares the performance of several classifier algorithms on a standard database of handwritten digits. We consider not only raw accuracy, but also training time, recognition time, and memory requirements. When available, we report measurements of the fraction of patterns that must be rejected so that the remaining patterns have misclassification rates less than a given threshold

Management Science, 1972

IEEE Transactions on Neural Networks, 1997

Pattern classification using neural networks and statistical methods is discussed. We give a tutorial overview in which popular classifiers are grouped into distinct categories according to their underlying mathematical principles; also, we assess what makes a classifier neural. The overview is complemented by two case studies using handwritten digit and phoneme data that test the performance of a number of most typical neural-network and statistical classifiers. Four methods of our own are included: Reduced kernel discriminant analysis, the learning -nearest neighbors classifier, the averaged learning subspace method, and a version of kernel discriminant analysis. , in 1993. His current research interests include neural networks, pattern recognition, as well as computational and mathematical statistics, in particular nonparametric function estimation. His earlier work has also included functional analysis, geometric modeling, and computer aided design.

ESAIM: Probability and Statistics, 2005

The last few years have witnessed important new developments in the theory and practice of pattern classification. We intend to survey some of the main new ideas that have led to these recent results.

2008

This article is devoted to experimental investigation of classification algorithms for analysis of ITS dataset. We introduce and consider a novel k-committees algorithm for classification and compare it with the discrete k-means and Nearest Neighbour algorithms. The ITS dataset consists of nuclear ribosomal DNA sequences, where rather sophisticated alignment scores have to be used as a measure of distance. These scores do not form a Minkowski metric and the sequences cannot be regarded as points in a finite dimensional space. This is why it is necessary to develop novel algorithms and adjust familiar ones. We present the results of experiments comparing the efficiency of three classification methods in their ability to achieve agreement with classes published in the biological literature before. It turns out that our algorithms are efficient and can be used to obtain biologically significant classifications. A simplified version of a synthetic dataset, where the k-committees classifier outperforms k-means and Nearest Neighbour classifiers, is also presented.