Regularized Jacobi Wavelets Kernel for Support Vector Machines

Journal of Industrial and Management Optimization, 2009

The Dual-nu Support Vector Machine (SVM) is an effective method in pattern recognition and target detection. It improves on the Dual-C SVM, and offers competitive performance in detection and computation with traditional classifiers. We show that the regularisation parameters Dual-nu and Dual-C can be set such that the same SVM solution is obtained. We present the process of determining the related parameters of one form from the solution of a trained SVM of the other form, and test the relationship with a digit recognition problem. The link between the Dual-nu and Dual-C parameters allows users to use Dual-nu for ease of training, and to switch between the two forms readily.

2004

1 11.1 Overview In this lecture we will describe methods for performing a binary classification task on a linearly non-separable data by the means of linear classification. We first explore the linear problem and the mathematical methods used to solve this problem. We then perform a generalization that will allow us to deal with more complex, noisy or non-linear situations, by embedding the input data into a higher dimensional feature-space, in which the data is separable (the concept is demonstrated in Figure 11.1). This will be accomplished a “kernel trick”

International Conference on Acoustics, Speech, and Signal Processing, 2003

Support vector machines (SVMs) are the most well known nonlinear classifiers based on the Mercer kernel trick. They generally lead to very sparse solutions that ensure good generalization performance. Recently, S. Mika et al. (see Advances in Neural Networks for Signal Processing, p.41-8, 1999) proposed a new nonlinear technique based on the kernel trick and the Fisher criterion: the nonlinear

2001

In this paper we give necessary and sufficient conditions under which kernels of dot product type k(x, y) = k(x . y) satisfy Mercer's condition and thus may be used in Support Vector Machines (SVM), Regularization Networks (RN) or Gaussian Processes (GP). In particular, we show that if the kernel is analytic (i.e. can be expanded in a Taylor series), all expansion coefficients have to be nonnegative. We give an explicit functional form for the feature map by calculating its eigenfunctions and eigenvalues.

2005

In this paper, we present a new compactly supported kernel for SVM based image recognition. This kernel which we called Geometric Compactly Supported (GCS) can be viewed as a generalization of spherical kernels to higher dimensions. The construction of the GCS kernel is based on a geometric approach using the intersection volume of two n-dimensional balls. The compactness property of the GCS kernel leads to a sparse Gram matrix which enhances computation efficiency by using sparse linear algebra algorithms. Comparisons of the GCS kernel performance, for image recognition task, with other known kernels prove the interest of this new kernel.

Lecture Notes in Computer Science, 2004

This paper contrasts three related regularization schemes for kernel machines using a least squares criterion, namely Tikhonov and Ivanov regularization and Morozov's discrepancy principle. We derive the conditions for optimality in a least squares support vector machine context (LS-SVMs) where they differ in the role of the regularization parameter. In particular, the Ivanov and Morozov scheme express the trade-off between data-fitting and smoothness in the trust region of the parameters and the noise level respectively which both can be transformed uniquely to an appropriate regularization constant for a standard LS-SVM. This insight is employed to tune automatically the regularization constant in an LS-SVM framework based on the estimated noise level, which can be obtained by e.g. by a nonparametric differogram technique.

Neural Networks, IEEE …, 2010

In this brief we have proposed the multiclass data classification by computationally inexpensive discriminant analysis through vector-valued regularized kernel function approximation (VVRKFA). VVRKFA being an extension of fast regularized kernel function approximation (FRKFA), provides the vector-valued response at single step. The VVRKFA finds a linear operator and a bias vector by using a reduced kernel that maps a pattern from feature space into the low dimensional label space. The classification of patterns is carried out in this low dimensional label subspace. A test pattern is classified depending on its proximity to class centroids. The effectiveness of the proposed method is experimentally verified and compared with multiclass support vector machine (SVM) on several benchmark data sets as well as on gene microarray data for multi-category cancer classification. The results indicate the significant improvement in both training and testing time compared to that of multiclass SVM with comparable testing accuracy principally in large data sets. Experiments in this brief also serve as comparison of performance of VVRKFA with stratified random sampling and sub-sampling.

2002

Taking advantage of the linear properties in high dimensional spaces, a general kind of kernel machines is formulated under a unified framework. These methods include KPCA, KFD and SVM. The theoretical framework will show a strong connection between KFD and SVM. The main practical result under the proposed framework is the solution of KFD for an arbitrary number of classes. The framework allows also the formulation of multiclass-SVM. The main goal of this article is focused in finding new solutions and not in the optimization of them.

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000

Wavelet kernels have been introduced for both support vector regression and classification. Most of these wavelet kernels do not use the inner product of the embedding space, but use wavelets in a similar fashion to radial basis function kernels. Wavelet analysis is typically carried out on data with a temporal or spatial relation between consecutive data points. We argue that it is possible to order the features of a general data set so that consecutive features are statistically related to each other, thus enabling us to interpret the vector representation of an object as a series of equally or randomly spaced observations of a hypothetical continuous signal. By approximating the signal with compactly supported basis functions and employing the inner product of the embedding L 2 space, we gain a new family of wavelet kernels. Empirical results show a clear advantage in favor of these kernels.

2004

The support vector machine (SVM) has been extended to build up nonlinear classifiers using the kernel trick. As a learning model, it has the best recognition performance among the many methods currently known because it is devised to obtain high performance for unlearned data. This paper reviews how to enhance generalization in learning classifiers centering on the SVM.