Johan Suykens

Followers

168

Following

Co-authors

Public Views

Uploads

Papers by Johan Suykens

Sparse least squares support vector machine classifiers

Neural Processing Letters, 2000

Since the early 90's, Support Vector Machines (SVM) are attracting more and more attention due to... more Since the early 90's, Support Vector Machines (SVM) are attracting more and more attention due to their applicability to a large number of problems. To overcome the high computational complexity of traditional Support Vector Machines, recently a new technique, the Least-Squares SVM (LS-SVM) has been introduced, but unfortunately a very attractive feature of SVM, namely its sparseness, was lost. LS-SVM simplifies the required computation to solving linear equation set. This equation set embodies all available information about the learning process. By applying modifications to this equation set, we present a Least Squares version of the Least Squares Support Vector Machine (LS 2-SVM). The proposed modification speeds up the calculations and provides better results, but most importantly it concludes a sparse solution.

Download

Improved Long-Term Temperature Prediction by Chaining of Neural Networks

Ijns, 2001

When an artificial neural network (ANN) is trained to predict signals p steps ahead, the quality ... more When an artificial neural network (ANN) is trained to predict signals p steps ahead, the quality of the prediction typically decreases for large values of p. In this paper, we compare two methods for prediction with ANNs: the classical recursion of one-step ahead predictors and a new kind of chain structure. When applying both techniques to the prediction of the temperature at the end of a blast furnace, we conclude that the chaining approach leads to an improved prediction of the temperature and avoidance of instabilities, since the chained networks gradually take the prediction of their predecessors in the chain as an extra input. It is observed that instabilities might occur in the iterative case, which does not happen with the chaining approach. To select relevant inputs and decrease the number of weights in this approach, Automatic Relevance Determination (ARD) for multilayer perceptrons is applied.

Interpolation Based MPC for LPV Systems using Polyhedral Invariant Sets

Proceedings of the 2005, American Control Conference, 2005., 2005

Guaranteeing asymptotic stability and recursive constraint satisfaction for a set of initial stat... more Guaranteeing asymptotic stability and recursive constraint satisfaction for a set of initial states that is as large as possible and with both a minimal control cost and computational load can be identified as a common objective in the Model Predictive Control (MPC) community. General interpolation provides a favourable trade off between these different aspects, however, in the robust case, this requires on-line Semi-Definite Programming (SDP), since one typically employs ellipsoidal invariant sets. Recently, have proposed an efficient algorithm for constructing the robust polyhedral maximal admissible set for linear systems with polytopic model uncertainty. In this paper a robust interpolation based MPC method is proposed that makes use of these sets. The algorithm is formulated as a Quadratic Program (QP) and is shown to have improved feasibility properties, efficiently cope with non-symmetrical constraints and give better control performance than existing interpolation based robust MPC algorithms. provides a favourable trade off between these different aspects, however, in the robust case, this requires on-line Semi-Definite Programming (SDP), since one typically employs ellipsoidal invariant sets. Recently, have proposed an efficient algorithm for constructing the robust polyhedral maximal admissible set for linear systems with polytopic model uncertainty. In this paper a robust interpolation based MPC method is proposed that makes use of these sets. The algorithm is formulated as a Quadratic Program (QP) and is shown to have improved feasibility properties, efficiently cope with non-symmetrical constraints and give better control performance than existing interpolation based robust MPC algorithms.

Download

Improved Long-Term Temperature Prediction by Chaining of Neural Networks

International Journal of Neural Systems, 2001

Hierarchical Semi-Supervised Clustering using KSC based model

by Siamak Mehrkanoon and Johan Suykens

— This paper introduces a methodology to incorporate the label information in discovering the und... more — This paper introduces a methodology to incorporate the label information in discovering the underlying clusters in a hierarchical setting using multi-class semi-supervised clustering algorithm. The method aims at revealing the relationship between clusters given few labels associated to some of the clusters. The problem is formulated as a regularized kernel spectral clustering algorithm in the primal-dual setting. The available labels are incorporated in different levels of hierarchy from top to bottom. As we advance towards the lowers levels in the tree all the previously added labels are used in the generation of the new levels of hierarchy. The model is trained on a subset of the data and then applied to the rest of the data in a learning framework. Thanks to the previously learned model, the out-of-sample extension property of the model allows then to predict the memberships of a new point. A combination of an internal clustering quality index and classification accuracy is used for model selection. Experiments are conducted on synthetic data and real image segmentation problems to show the applicability of the proposed approach.

Download

Large scale semi-supervised learning using KSC based model

by Siamak Mehrkanoon and Johan Suykens

— Often in practice one deals with a large amount of unlabeled data, while the fraction of labele... more — Often in practice one deals with a large amount of unlabeled data, while the fraction of labeled data points will typically be small. Therefore one prefers to apply a semi-supervised algorithm, which uses both labeled and unlabeled data points in the learning process, to have a better performance. Considering the large amount of unlabeled data, making a semi-supervised algorithm scalable is an important task. In this paper we adopt a recently proposed multi-class semi-supervised KSC based algorithm (MSS-KSC) and make it scalable by means of two different approaches. The first one is based on the Nyström approximation method which provides a finite dimensional feature map that can then be used to solve the optimization problem in the primal. The second approach is based on the reduced kernel technique that solves the problem in the dual by reducing the dimensionality of the kernel matrix to a rectangular kernel. Experimental results demonstrate the scalability and efficiency of the proposed approaches on real datasets.

Download

Symbolic computing of LS-SVM based models

by Johan Suykens and Siamak Mehrkanoon

This paper introduces a software tool SYM-LS-SVM-SOLVER written in Maple to derive the dual syste... more This paper introduces a software tool SYM-LS-SVM-SOLVER written in Maple to derive the dual system and the dual model representation of LS-SVM based models, symbolically. SYM-LS-SVM-SOLVER constructs the Lagrangian from the given objective function and list of constraints. Afterwards it obtains the KKT (Karush-Kuhn-Tucker) optimality conditions and finally formulates a linear system in terms of the dual variables. The effectiveness of the developed solver is illustrated by applying it to a variety of problems involving LS-SVM based models.

Download

Incremental multi-class semi-supervised clustering regularized by Kalman filtering

by Siamak Mehrkanoon, Oscar Agudelo, and Johan Suykens

This paper introduces an on-line semi-supervised learning algorithm formulated as a regularized k... more This paper introduces an on-line semi-supervised learning algorithm formulated as a regularized kernel spectral clustering (KSC) approach. We consider the case where new data arrive sequentially but only a small fraction of it is labeled. The available labeled data act as prototypes and help to improve the performance of the algorithm to estimate the labels of the unlabeled data points. We adopt a recently proposed multi-class semi-supervised KSC based algorithm (MSS-KSC) and make it applicable for on-line data clustering. Given a few user-labeled data points the initial model is learned and then the class membership of the remaining data points in the current and subsequent time instants are estimated and propagated in an on-line fashion. The update of the memberships is carried out mainly using the out-of-sample extension property of the model. Initially the algorithm is tested on computer-generated data sets, then we show that video segmentation can be cast as a semi-supervised learning problem. Furthermore we show how the tracking capabilities of the Kalman filter can be used to provide the labels of objects in motion and thus regularizing the solution obtained by the MSS-KSC algorithm. In the experiments, we demonstrate the performance of the proposed method on synthetic data sets and real-life videos where the clusters evolve in a smooth fashion over time.

Download

Multi-class Semi-supervised Learning based Upon Kernel Spectral Clustering

by Siamak Mehrkanoon and Johan Suykens

—This paper proposes a multi-class semi-supervised learning algorithm using kernel spectral clust... more —This paper proposes a multi-class semi-supervised learning algorithm using kernel spectral clustering (KSC) as a core model. A regularized KSC is formulated to estimate the class memberships of data points in a semi-supervised setting using the one-vs-all strategy while both labeled and unlabeled data points are present in the learning process. The propagation of the labels to a large amount of unlabeled data points is achieved by adding the regularization terms to the cost function of the KSC formulation. In other words, imposing the regularization term enforces certain desired memberships. The model is then obtained by solving a linear system in the dual. Furthermore, the optimal embedding dimension is designed for semi-supervised clustering. This plays a key role when one deals with a large number of clusters. Index Terms—Semi-supervised learning, kernel spectral clustering , low embedding dimension for clustering, multi-class problem.

Download

Sparse Reductions For Fixed-Size Least Squares Supoort Vector Machines on Large Scale Data

by Johan Suykens and Raghvendra Mall

Ranking Overlap and Outlier Points in Data using Soft Kernel Spectral Clustering

by Johan Suykens, Raghvendra Mall, and Rocco Langone

Soft clustering algorithms can handle real-life datasets better as they capture the presence of i... more Soft clustering algorithms can handle real-life datasets better as they capture the presence of inherent overlapping clusters. A soft kernel spectral clustering (SKSC) method proposed in [1] exploited the eigen-projections of the points to assign them different cluster membership probabilities. In this paper, we detect points in dense overlapping regions as overlap points. We also identify the outlier points by exploiting the eigen-projections. We then propose novel ranking techniques using structure and similarity properties in the eigen-space to rank these overlap and outlier points. By ranking the overlap and outlier points we provide an order for the most and least influential points in the dataset. We demonstrate the effectiveness of our ranking measures on several datasets.

Download

Does the combination of magnetic resonance imaging and spectroscopic imaging improve the classification of brain tumours?

Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference, 2004

Magnetic Resonance Imaging (MRI) and magnetic resonance spectroscopic imaging (MRSI) play an impo... more Magnetic Resonance Imaging (MRI) and magnetic resonance spectroscopic imaging (MRSI) play an important role in the noninvasive diagnosis of brain tumours. We investigate the use of both MRI and MRSI, separately and in combination with each other for classification of brain tissue types. Many clinically relevant classification problems are considered; for example healthy versus tumour tissues, low- versus high-grade tumours. Linear as well as nonlinear techniques are compared. The classification performance is evaluated by the area under the receiver operating characteristic (ROC) curve (AUC). In general, all techniques achieve a high performance, except when using MRI alone. For example, for low- versus high-grade tumours, low- versus high-grade gliomas, gliomas versus meningiomas, respectively a test AUC higher than 0.91, 0.93 and 0.98 is reached, when both MRI and MRSI data are used.

Regularized and sparse stochastic k-means for distributed large-scale clustering

2015 IEEE International Conference on Big Data (Big Data), 2015

Journal of Machine Learning Research 1 (2003) 1-48 Submitted 4/00; Published 10/00 Coupled Transductive Ensemble Learning of

ABSTRACT

A Weighted Kernel PCA Formulation with Out-of-Sample Extensions for Spectral Clustering Methods

The 2006 IEEE International Joint Conference on Neural Network Proceedings, 2000

ABSTRACT

Robust Nonlinear H1 Synchronization

ABSTRACT

A PARTAN-Accelerated Frank-Wolfe Algorithm for Large-Scale SVM Classification

Frank-Wolfe algorithms have recently regained the attention of the Machine Learning community. Th... more Frank-Wolfe algorithms have recently regained the attention of the Machine Learning community. Their solid theoretical properties and sparsity guarantees make them a suitable choice for a wide range of problems in this field. In addition, several variants of the basic procedure exist that improve its theoretical properties and practical performance. In this paper, we investigate the application of some of these techniques to Machine Learning, focusing in particular on a Parallel Tangent (PARTAN) variant of the FW algorithm that has not been previously suggested or studied for this type of problems. We provide experiments both in a standard setting and using a stochastic speed-up technique, showing that the considered algorithms obtain promising results on several medium and large-scale benchmark datasets for SVM classification.

Download

Alarm prediction in industrial machines using autoregressive LS-SVM models

2014 Ieee Symposium on Computational Intelligence and Data Mining, Dec 1, 2014

In industrial machines different alarms are embedded in machines controllers. They make use of se... more In industrial machines different alarms are embedded in machines controllers. They make use of sensors and machine states to indicate to end-users various information (e.g. diagnostics or need of maintenance) or to put machines in a specific mode (e.g. shut-down when thermal protection is activated). More specifically, the alarms are often triggered based on comparing sensors data to a threshold defined in the controllers software. In batch production machines, triggering an alarm (e.g. thermal protection) in the middle of a batch production is crucial for the quality of the produced batch and results into a high production loss. This situation can be avoided if the settings of the production machine (e.g. production speed) is adjusted accordingly based on the temperature monitoring. Therefore, predicting a temperature alarm and adjusting the production speed to avoid triggering the alarm seems logical. In this paper we show the effectiveness of Least Squares Support Vector Machines (LS-SVMs) in predicting the evolution of the temperature in a steel production machine and, as a consequence, possible alarms due to overheating. Firstly, in an offline fashion, we develop a nonlinear autoregressive (NAR) model, where a systematic model selection procedure allows to carefully tune the model parameters. Afterwards, the NAR model is used online to forecast the future temperature trend. Finally, a classifier which uses as input the outcomes of the NAR model allows to foresee future alarms.

Representative Subsets For Big Data Learning using kNN graphs

by Rocco Langone, Johan Suykens, and Raghvendra Mall

In this paper we propose a deterministic method to obtain subsets from big data which are a good ... more In this paper we propose a deterministic method to obtain subsets from big data which are a good representative of
the inherent structure in the data. We first convert the large scale dataset into a sparse undirected k-NN graph using a
distributed network generation framework that we propose in this paper. After obtaining the k-NN graph we exploit the fast
and unique representative subset (FURS) selection method to deterministically obtain a subset for this big data network.
The FURS selection technique selects nodes from different dense regions in the graph retaining the natural community structure. We then locate the points in the original big data corresponding to the selected nodes and compare the obtained subset with subsets acquired from state-of-the-art subset selection techniques. We evaluate the quality of the selected subset on several synthetic and real-life datasets for different learning tasks including big data classification and big data clustering.

Download

Kernel Spectral Clustering for dynamic data using Multiple Kernel Learning

by Diego Hernán Peluffo-Ordóñez, Rocco Langone, and Johan Suykens

In this paper we propose a kernel-spectralclustering-based technique to catch the different regim... more In this paper we propose a kernel-spectralclustering-based technique to catch the different regimes experienced by a time-varying system. Our method is based on a multiple kernel learning approach, which is a linear combination of kernels. The calculation of the linear combination coefficients is done by determining a ranking vector that quantifies the overall dynamical behavior of the data sequence analyzed over time. This vector can be calculated from the eigenvectors provided by the solution of the kernel spectral clustering problem. We apply the proposed technique to a trial from the Graphics Lab Motion Capture Database from Carnegie Mellon University, as well as to a synthetic database, namely three moving Gaussian clouds. For comparison purposes, some conventional spectral clustering techniques are also considered, namely, kernel kmeans, min-cuts as well as standard k-means. The normalized mutual information and adjusted random index metrics are used to quantify the clustering performance. Results show the usefulness and applicability of proposed technique to track dynamic data, even being able to detect hidden objects.

Download

Johan Suykens

Uploads

Papers by Johan Suykens

Log In