The GCS kernel for SVM-based image recognition

2012

In this paper, A novel classification approach based on sparse representation framework is proposed. The method finds the minimum Euclidian distance between an input patch (pattern) and atoms (templates) of a learnt-base dictionary for different classes to perform the classification task. A mathematical approach is developed to map the sparse representation vector to Euclidian distances. We show that the highest coefficient of the sparse vector is not necessarily a suitable indicator to classify input patches, and it results in classification errors. The K-SVD dictionary learning method is utilized to separately create class specific sub-dictionaries. The proposed algorithm is compared with the conventional sparse representation classification (SRC) framework to evaluate its performance. Our experimental results demonstrate a higher accuracy with a lower computational time.

2007

Kernel classifiers based on Support Vector Machines (SVM) have recently achieved state-of-the art results on several popular datasets like Caltech or Pascal. This was possible by combining the advantages of SVM -convexity and the availability of efficient optimizers, with 'hyperkernels' -linear combinations of kernels computed at multiple levels of image encoding. The use of hyperkernels faces the challenge of choosing the kernel weights, the use of possibly irrelevant, poorly performing kernels, and an increased number of parameters that can lead to overfitting. In this paper we advocate the transition from SVMs to Support Kernel Machines (SKM) -models that estimate both the parameters of a sparse linear combination of kernels, and the parameters of a discriminative classifier. We exploit recent kernel learning techniques, not previously used in computer vision, that show how learning SKMs can be formulated as a convex optimization problem, which can be solved efficiently using Sequential Minimal Optimization. We study kernel learning for several multi-level image encodings for supervised object recognition and report competitive results on several datasets, including INRIA pedestrian, Caltech 101 and the newly created Caltech 256.

Journal of Mathematical Imaging and Vision

This paper presents a supervised subspace learning method called Kernel Generalized Discriminative Common Vectors (KGDCV), as a novel extension of the known Discriminative Common Vectors method with Kernels. Our method combines the advantages of kernel methods to model complex data and solve nonlinear problems with moderate computational complexity, with the better generalization properties of generalized approaches for large dimensional data. These attractive combination makes KGDCV specially suited for feature extraction and classification in computer vision, image processing and pattern recognition applications. Two different approaches to this generalization are proposed, a first one based on the kernel trick (KT) and a second one based on the nonlinear projection trick (NPT) for even higher efficiency. Both methodologies have been validated on four different image datasets containing faces, objects and handwritten digits, and compared against well known non-linear state-of-art methods. Results show better discriminant properties than other generalized approaches both linear or kernel. In addition, the KGDCV-NPT approach presents a considerable computational gain, without compromising the accuracy of the model.

2004

This paper investigates the effect of Kernel Principal Component Analysis (KPCA) within the classification framework, essentially the regularization properties of this dimensionality reduction method. KPCA has been previously used as a pre-processing step before applying an SVM but we point out that this method is somewhat redundant from a regularization point of view and we propose a new algorithm called Kernel Projection Machine to avoid this redundancy, based on an analogy with the statistical framework of regression for a Gaussian white noise model. Preliminary experimental results show that this algorithm reaches the same performances as an SVM.

2012 Visual Communications and Image Processing, 2012

In this paper, A novel classification approach based on sparse representation framework is proposed. The method finds the minimum Euclidian distance between an input patch (pattern) and atoms (templates) of a learnt-base dictionary for different classes to perform the classification task. A mathematical approach is developed to map the sparse representation vector to Euclidian distances. We show that the highest coefficient of the sparse vector is not necessarily a suitable indicator to classify input patches, and it results in classification errors. The K-SVD dictionary learning method is utilized to separately create class specific sub-dictionaries. The proposed algorithm is compared with the conventional sparse representation classification (SRC) framework to evaluate its performance. Our experimental results demonstrate a higher accuracy with a lower computational time.

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000

Wavelet kernels have been introduced for both support vector regression and classification. Most of these wavelet kernels do not use the inner product of the embedding space, but use wavelets in a similar fashion to radial basis function kernels. Wavelet analysis is typically carried out on data with a temporal or spatial relation between consecutive data points. We argue that it is possible to order the features of a general data set so that consecutive features are statistically related to each other, thus enabling us to interpret the vector representation of an object as a series of equally or randomly spaced observations of a hypothetical continuous signal. By approximating the signal with compactly supported basis functions and employing the inner product of the embedding L 2 space, we gain a new family of wavelet kernels. Empirical results show a clear advantage in favor of these kernels.

2020

Support Vector Machine (SVM) is a powerful paradigm that has proven to be extremely useful for the task of classifying high-dimensional objects. It does not only perform well in learning linear classifiers, but also shows outstanding performance in capturing nonlinearity through the use of kernels. In principle, SVM allows us to train “scoring” classifiers i.e. classifiers that output a prediction score. However, it can also be adapted to produce probability-type outputs through the use of the Venn-Abers framework. This allows us to obtain valuable information on the labels distribution for each test object. This procedure, however, is restricted to very small data given its inherent computational complexity. We circumvent this limitation by borrowing results from the field of computational geometry. Specifically, we make use of the concept of a coreset: a small summary of data that is constructed by discretising the input space into enclosing balls, so that each ball will be repres...

We describe an approach to kernel selection in Support Vector Machines (SVMs) driven by the Gram matrix. Our study extracts properties from this matrix (e.g., Fisher's discriminant, Bregman's divergence) using different kernel functions (lin-ear, polynomial, Gaussian, Laplacian, Bessel and ANOVARBF), and incorporates such properties as meta-features within a meta-learning framework. The goal is to predict the best kernel in SVMs. Results show how introducing a new meta-feature, Distance Ratio, capturing inter-class and intra-class distances in the fea-ture space, yields substantial improvements during kernel selection.

IEEE International Conference on Image Processing 2005, 2005

Histogram Intersection (HI) kernel has been recently introduced for image recognition tasks. The HI kernel is proved to be positive definite and thus can be used in Support Vector Machine (SVM) based recognition. Experimentally, it also leads to good recognition performances. However, its derivation applies only for binary strings such as color histograms computed on equally sized images. In this paper, we propose a new kernel, which we named Generalized Histogram Intersection (GHI) kernel, since it applies in a much larger variety of contexts. First, an original derivation of the positive definiteness of the GHI kernel is proposed in the general case. As a consequence, vectors of real values can be used, and the images no longer need to have the same size. Second, a hyper-parameter is added, compared to the HI kernel, which allows us to better tune the kernel model to particular databases. We present experiments which prove that the GHI kernel outperforms the simple HI kernel in a simple recognition task. Comparisons with other wellknown kernels are also provided.

2020

This paper is an announcement for our longer paper in preparation. Traditional kernel based methods utilize either a fixed kernel or a combination of judiciously chosen kernels from a fixed dictionary. In contrast, we construct a datadependent kernel utilizing the components of the eigen-decompositions of different kernels constructed using ideas from diffusion geometry, and use a regularization technique with this kernel with adaptively chosen parameters. In this paper, we illustrate our method using the two moons dataset, where we obtain a zero test error using only a minimal number of training samples.