Learning k-Nearest Neighbor Naive Bayes for Ranking

Machine Learning: ECML 2000, 2000

We investigate the problem of using past performance information to select an algorithm for a given classi cation problem. We present three ranking methods for that purpose: average ranks, success rate ratios and signi cant wins. We also analyze the problem of evaluating and comparing these methods. The evaluation technique used is based on a leave-one-out procedure. On each iteration, the method generates a ranking using the results obtained by the algorithms on the training datasets. This ranking is then evaluated by calculating its distance from the ideal ranking built using the performance information on the test dataset. The distance measure adopted here, average correlation, is based on Spearman's rank correlation coe cient. To compare ranking methods, a combination of Friedman's test and Dunn's multiple comparison procedure is adopted. When applied to the methods presented here, these tests indicate that the success rate ratios and average ranks methods perform better than signi cant wins.

2008

Many ranking models have been proposed in information retrieval, and recently machine learning techniques have also been applied to ranking model construction. Most of the existing methods do not take into consideration the fact that significant differences exist between queries, and only resort to a single function in ranking of documents. In this paper, we argue that it is necessary to employ different ranking models for different queries and conduct what we call query-dependent ranking. As the first such attempt, we propose a K-Nearest Neighbor (KNN) method for querydependent ranking. We first consider an online method which creates a ranking model for a given query by using the labeled neighbors of the query in the query feature space and then rank the documents with respect to the query using the created model. Next, we give two offline approximations of the method, which create the ranking models in advance to enhance the efficiency of ranking. And we prove a theory which indicates that the approximations are accurate in terms of difference in loss of prediction, if the learning algorithm used is stable with respect to minor changes in training examples. Our experimental results show that the proposed online and offline methods both outperform the baseline method of using a single ranking function.

2004

Accurate ranking is desired in many real-world data mining applications. Traditional learning algorithms, however, aim only at high classification accuracy. It has been observed that both traditional decision trees and naive Bayes produce good classification accuracy but poor probability estimates. In this paper, we use a new model, conditional independence tree (CITree), which is a combination of decision tree and naive Bayes and more suitable for ranking and more learnable in practice. We propose a novel algorithm for learning CITree for ranking, and the experiments show that the CITree algorithm outperforms the stateof-the-art decision tree learning algorithm C4.4 and naive Bayes significantly in yielding accurate rankings. Our work provides an effective data mining algorithm for applications in which an accurate ranking is required.

Pattern Recognition Letters, 2012

Some new rank methods to select the best prototypes from a training set are proposed in this paper in order to establish its size according to an external parameter, while maintaining the classification accuracy. The traditional methods that filter the training set in a classification task like editing or condensing have some rules that apply to the set in order to remove outliers or keep some prototypes that help in the classification. In our approach, new voting methods are proposed to compute the prototype probability and help to classify correctly a new sample. This probability is the key to sorting the training set out, so a relevance factor from 0 to 1 is used to select the best candidates for each class whose accumulated probabilities are less than that parameter. This approach makes it possible to select the number of prototypes necessary to maintain or even increase the classification accuracy. The results obtained in different high dimensional databases show that these methods maintain the final error rate while reducing the size of the training set.

2003

The k-Nearest-Neighbours (kNN) is a simple but effective method for classification. The major drawbacks with respect to kNN are (1) its low efficiency – being a lazy learning method prohibits it in many applications such as dynamic web mining for a large repository, and (2) its dependency on the selection of a “good value” for k. In this paper, we propose a novel kNN type method for classification that is aimed at overcoming these shortcomings. Our method constructs a kNN model for the data, which replaces the data to serve as the basis of classification. The value of k is automatically determined, is varied for different data, and is optimal in terms of classification accuracy. The construction of the model reduces the dependency on k and makes classification faster. Experiments were carried out on some public datasets collected from the UCI machine learning repository in order to test our method. The experimental results show that the kNN based model compares well with C5.0 and kNN in terms of classification accuracy, but is more efficient than the standard kNN.

2005

Different ways of contrast generated rankings by feature selection algorithms are presented in this paper, showing several possible interpretations, depending on the given approach to each study. We begin from the premise of no existence of only one ideal subset for all cases. The purpose of these kinds of algorithms is to reduce the data set to each first attributes without losing prediction against the original data set. In this paper we propose a method, feature-ranking performance, to compare different feature-ranking methods, based on the Area Under Feature Ranking Classification Performance Curve (AURC). Conclusions and trends taken from this paper propose support for the performance of learning tasks, where some ranking algorithms studied here operate.

2009 IEEE 25th International Conference on Data Engineering, 2009

In this paper, we formalize the novel concept of incremental reverse nearest neighbor ranking and suggest an original solution for this problem. We propose an efficient approach for reporting the results incrementally without the need to restart the search from scratch. Our approach can be applied to a multi-dimensional feature database which is hierarchically organized by any R-tree like index structure. Our solution does not assume any preprocessing steps which makes it applicable for dynamic environments where updates of the database frequently occur. Our experiments show that our approach reports the ranking results with much less page accesses than existing approaches designed for traditional reverse nearest neighbor search applied to the ranking problem.

2008

Abstract Given a binary classification task, a ranker is an algorithm that can sort a set of instances from highest to lowest expectation that the instance is positive. In contrast to a classifier, a ranker does not output class predictions ��������although it can be turned into a classifier with help of an additional procedure to split the ranked list into two. A straightforward way to compute rankings is to train a scoring classifier to assign numerical scores to instances, for example the predicted odds that an instance is positive.

Research Inventy - International Journal of Engineering and Science, 2019

Order is the way toward finding (or preparing) an arrangement of models (or capacities) that portray and recognize information classes or ideas. That is to be ready to utilize the models to foresee the obscure class marks of cases. We manage the positioning issue in this proposal. The positioning issue is an exceptional instance of the classification issue, where the class marks are positions or appraisals, spoke to by numbers from 1 to q. The positioning issue can likewise be given a role as the way toward preparing a rank-forecast show that appoints each example a rank that is as close as could be expected under the circumstances "to the occurrence's genuine rank. Well known uses of the positioning issue incorporate positioning the significance of website pages, assessing the financial credit of a man, and positioning the dangers of speculations. Two mainstream groups of strategies to take care of positioning issues are Multi-Criteria Decision Aid (MCDA) techniques and Support Vector Machines (SVMs). The execution of effective MCDA techniques, for example, Utilities Additives Discriminates (UTADIS) and Generalized Utilities Additives Discriminates (GUTADIS), is accomplished by abusing the foundation information that de-copyists the relationships between are the qualities and the positions. Lamentably, the foundation information is case-subordinate, thus it is probably going to be unveil capable, vague or difficult to be demonstrated by and by. This limits the application of MCDA techniques. SVMs, rather, don't require any foundation learning. Their great execution is accomplished by keeping balance between limiting the observational misfortune and augmenting the partition edge. Normally, a multi-class Support Vector Machine Classifier is developed by combining a few twofold Support Vector Machine Classifiers. In the SVM-based approach the positioning data isn't utilized. This proposition endeavors to build an efficient calculation for positioning issues. We look at the properties of existing calculations for positioning problems and propose a half breed calculation that joins the multi-class SVM (M-SVM) and the UTADIS demonstrate. In the new calculation, the double SVM classifiers are joined into a multi-class classifier in light of the fluffy voting strategy. The ideal fluffy voting technique is sought by settling a Linear Program (LP). The new calculation is called Fuzzy Voting based Support Vector Ranking (FVSVR) technique. We likewise expand the possibility of Fuzzy Voting from positioning issues to nonexclusive multi-class classification issues, which prompts a Fuzzy Voting based Support Vector Machine (FVSVM) strategy. The benefits of FVSVR and FVSVM are exhibited by trial comes about in view of a few databases of handy classification issues.

2007

The estimation of the quality of attributes is an important issue in machine learning and data mining. There are several important tasks in the process of machine learning like feature subset selection, constructive induction, and decision tree building, which contain the attribute estimation procedure as their principal component. Relief algorithms are successful attribute estimators. They are able to detect conditional dependencies between attributes and provide a unified view on the attribute estimation. They have commonly been viewed as feature subset selection methods that are applied in pre-processing step before a model is learned. In this paper, we propose a variant of ReliefF algorithm: ReliefMSS. We analyse the ReliefMSS parameters and compare ReliefF and ReliefMSS performances as regards the number of iterations, the number of random attributes, the noise effect, the number of nearest neighbours and the number of examples presented. We find that for the most of these para...