IEEE Transactions on Emerging Topics in Computational Intelligence
Self-Organized Operational Neural Networks (Self-ONNs) have recently been proposed as new-generat... more Self-Organized Operational Neural Networks (Self-ONNs) have recently been proposed as new-generation neural network models with nonlinear learning units, i.e., the generative neurons that yield an elegant level of diversity; however, like its predecessor, conventional Convolutional Neural Networks (CNNs), they still have a common drawback: localized (fixed) kernel operations. This severely limits the receptive field and information flow between layers and thus brings the necessity for deep and complex models. It is highly desired to improve the receptive field size without increasing the kernel dimensions. This requires a significant upgrade over the generative neurons to achieve the "non-localized kernel operations" for each connection between consecutive layers. In this article, we present superior (generative) neuron models (or super neurons in short) that allow random or learnable kernel shifts and thus can increase the receptive field size of each connection. The kernel localization process varies among the two super-neuron models. The first model assumes randomly localized kernels within a range and the second one learns (optimizes) the kernel locations during training. An extensive set of comparative evaluations against conventional and deformable convolutional, along with the generative neurons demonstrates that super neurons can empower Self-ONNs to achieve a superior learning and generalization capability with a minimal computational complexity burden. PyTorch implementation of Self-ONNs with super-neurons is now publically shared. Index Terms-Convolutional neural networks, generative neurons, non-localized kernels, operational neural networks, receptive field. I. INTRODUCTION G ENERALIZED Operational Perceptrons (GOPs) [1], [2], [3], [4], [5] have been proposed as an advanced model of biological neurons with varying nonlinear synaptic connections. Thanks to such a diverse neuron model, GOPs have achieved a superior learning capability on many challenging problems surpassing conventional Multi-Layer Perceptrons (MLPs) and
Operational Neural Networks (ONNs) have recently been proposed to address the well-known limitati... more Operational Neural Networks (ONNs) have recently been proposed to address the well-known limitations and drawbacks of conventional Convolutional Neural Networks (CNNs) such as network homogeneity with the sole linear neuron model. ONNs are heterogenous networks with a generalized neuron model that can encapsulate any set of non-linear operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. However, Greedy Iterative Search (GIS) method, which is the search method used to find optimal operators in ONNs takes many training sessions to find a single operator set per layer. This is not only computationally demanding, but the network heterogeneity is also limited since the same set of operators will then be used for all neurons in each layer. Moreover, the performance of ONNs directly depends on the operator set library used, which introduces a certain risk of performance degradation especially when the optimal operator set required for a particular task is missing from the library. In order to address these issues and achieve an ultimate heterogeneity level to boost the network diversity along with computational efficiency, in this study we propose Selforganized ONNs (Self-ONNs) with generative neurons that have the ability to adapt (optimize) the nodal operator of each connection during the training process. Therefore, Self-ONNs can have an utmost heterogeneity level required by the learning problem at hand. Moreover, this ability voids the need of having a fixed operator set library and the prior operator search within the library in order to find the best possible set of operators. We further formulate the training method to back-propagate the error through the operational layers of Self-ONNs. Experimental results over four challenging problems demonstrate the superior learning capability and computational efficiency of Self-ONNs over conventional ONNs and CNNs, even with more compact networks.
The recently proposed network model, Operational Neural Networks (ONNs), can generalize the conve... more The recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) that are homogenous only with a linear neuron model. As a heterogenous network model, ONNs are based on a generalized neuron model that can encapsulate any set of non-linear operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. However, the default search method to find optimal operators in ONNs, the so-called Greedy Iterative Search (GIS) method, usually takes several training sessions to find a single operator set per layer. This is not only computationally demanding, also the network heterogeneity is limited since the same set of operators will then be used for all neurons in each layer. To address this deficiency and exploit a superior level of heterogeneity, in this study the focus is drawn on searching the best-possible operator set(s) for the hidden neurons of the network based on the "Synaptic Plasticity" paradigm that poses the essential learning theory in biological neurons. During training, each operator set in the library can be evaluated by their synaptic plasticity level, ranked from the worst to the best, and an "elite" ONN can then be configured using the top ranked operator sets found at each hidden layer. Experimental results over highly challenging problems demonstrate that the elite ONNs even with few neurons and layers can achieve a superior learning performance than GIS-based ONNs and as a result the performance gap over the CNNs further widens.
International Conference on Electrical and Electronics Engineering, Nov 1, 2017
In this study, three different feature selection algorithms are compared using Support Vector Mac... more In this study, three different feature selection algorithms are compared using Support Vector Machines as classifier for cancer classification through gene expression data. The ability of feature selection algorithms to select an optimal gene subset for a cancer type is evaluated by the classification ability of selected genes. A publicly available micro array dataset is employed for gene expression values. Selected gene subsets were able to classify subtypes of the considered cancer type with high accuracies and showed that these feature selection methods were applicable for bio-marker gene selection.
Application of evolutionary algorithms to garment design
In this study, we present the development of an intelligent system solution for fashion style sel... more In this study, we present the development of an intelligent system solution for fashion style selection for various female body shapes. The proposed intelligent system combines binary genetic algorithm (GA) or binary version of the particle swarm optimization (PSO) with PSO-trained artificial neural network. The former is used to search the solution space for the optimal design parameters corresponding to a best fit for the desired target, and the task of the latter is to evaluate fitness (goodness) of each evolved new fashion style. With the goal of creating natural aesthetic relationship between the shape of the body and the shape of the garment for fashion styling, combinations of upper body related and lower body related garment pieces together with detailed attribute categories were created as a knowledge base. The encouraging results of preliminary experiments demonstrate the feasibility of applying intelligent systems to fashion styling.
Particle Swarm Optimization (PSO) is attracting an ever-growing attention and more than ever it h... more Particle Swarm Optimization (PSO) is attracting an ever-growing attention and more than ever it has found many application areas for many challenging optimization problems. In this paper, we draw the focus on a major drawback of the PSO algorithm: the poor gbest update. This can be a severe problem, which causes premature convergence to local optima since gbest as the common term in the update equation of all particles, is the primary guide of the swarm. Therefore, we basically seek a solution for the social problem in PSO, i.e. "Who will guide the guide?" which resembles the rhetoric question posed by Plato in his famous work on government: "Who will guard the guards?" (Quis custodiet ipsos custodes?). Stochastic approximation (SA) is purposefully adapted into two approaches to guide (or drive) the gbest particle (with simultaneous perturbation) towards the right direction with the gradient estimate of the underlying surface (or function) whilst avoiding local traps due to its stochastic nature. We purposefully used simultaneous perturbation SA (SPSA) for its low cost and since SPSA is applied only to the gbest (not the entire swarm), both approaches have thus a negligible overhead cost over the entire PSO process. Yet we have shown over a wide range of non-linear functions that both approaches significantly improve the performance of PSO especially if the parameters of SPSA suits to the problem in hand.
Classification of polarimetric synthetic aperture radar (PolSAR) images is an active research are... more Classification of polarimetric synthetic aperture radar (PolSAR) images is an active research area with a major role in environmental applications. The traditional Machine Learning (ML) methods proposed in this domain generally focus on utilizing highly discriminative features to improve the classification performance, but this task is complicated by the well-known "curse of dimensionality" phenomena. Other approaches based on deep Convolutional Neural Networks (CNNs) have certain limitations and drawbacks, such as high computational complexity, an unfeasibly large training set with ground-truth labels, and special hardware requirements. In this work, to address the limitations of traditional ML and deep CNN based methods, a novel and systematic classification framework is proposed for the classification of PolSAR images, based on a compact and adaptive implementation of CNNs using a sliding-window classification approach. The proposed approach has three advantages. First, there is no requirement for an extensive feature extraction process. Second, it is computationally efficient due to utilized compact configurations. In particular, the proposed compact and adaptive CNN model is designed to achieve the maximum classification accuracy with minimum training and computational complexity. This is of considerable importance considering the high costs involved in labelling in PolSAR classification. Finally, the proposed approach can perform classification using smaller window sizes than deep CNNs. Experimental evaluations have been performed over the most commonly-used four benchmark PolSAR images: AIRSAR L-Band and RADARSAT-2 C-Band data of San Francisco Bay and Flevoland areas. Accordingly, the best obtained overall accuracies range between 92.33-99.39% for these benchmark study sites.
Th is paper presents a new measurement system of three dimensional coordinates based on a multi-l... more Th is paper presents a new measurement system of three dimensional coordinates based on a multi-link arm and a corresponding precise mathemat ical model. Model coefficients are estimated beforehand by the PSO method and the joint angle data which are obtained when the multi-lin k arm tip indicates known calibration coordinates. In the model, especially, deviations from the ideal position of rotation axis are considered. Such uncertainties are identified by offline supervised learning utilizing a PSO method. The proposed method can be regarded as a hybrid measurement method based on hardware and software. Therefore, mu lti-link 3-D measuring mach ine as a mechanical system can be produced easily without the need for ult ra-precision machining since mechanical uncertainties of the measuring device are built into the model and these will be estimated after offline learning. The effectiveness of the proposed measurement scheme is verified by simp le nu merical simu lation results.
This paper proposes an evolutionary RBF network classifier for polarimetric synthetic aperture ra... more This paper proposes an evolutionary RBF network classifier for polarimetric synthetic aperture radar (SAR) images. The proposed feature extraction process utilizes the full covariance matrix, the gray level co-occurrence matrix (GLCM) based texture features, and the backscattering power (Span) combined with the H/α/A decomposition, which are projected onto a lower dimensional feature space using principal component analysis. An experimental study is performed using the fully polarimetric San Francisco Bay data set acquired by the NASA/Jet Propulsion Laboratory Airborne SAR (AIRSAR) at L-band to evaluate the performance of the proposed classifier. Classification results (in terms of confusion matrix, overall accuracy and classification map) compared to the Wishart and a recent NN-based classifiers demonstrate the effectiveness of the proposed algorithm.
Hyperspectral image (HSI) classification is an important task in many applications, such as envir... more Hyperspectral image (HSI) classification is an important task in many applications, such as environmental monitoring, medical imaging, and land use/land cover (LULC) classification. Due to the significant amount of spectral information from recent HSI sensors, analyzing the acquired images is challenging using traditional Machine Learning (ML) methods. As the number of frequency bands increases, the required number of training samples increases exponentially to achieve a reasonable classification accuracy, also known as the curse of dimensionality. Therefore, separate band selection or dimensionality reduction techniques are often applied before performing any classification task over HSI data. In this study, we investigate recently proposed subspace learning methods for one-class classification (OCC). These methods map high-dimensional data to a lower-dimensional feature space that is optimized for one-class classification. In this way, there is no separate dimensionality reduction or feature selection procedure needed in the proposed classification framework. Moreover, one-class classifiers have the ability to learn a data description from the category of a single class only. Considering the imbalanced labels of the LULC classification problem and rich spectral information (high number of dimensions), the proposed classification approach is well-suited for HSI data. Overall, this is a pioneer study focusing on subspace learning-based one-class classification for HSI data. We analyze the performance of the proposed subspace learning one-class classifiers in the proposed pipeline. Our experiments validate that the proposed approach helps tackle the curse of dimensionality along with the imbalanced nature of HSI data.
Fashion Styling Recommendation with Intelligent Systems: Test of an Online Platform
Journal of fashion technology & textile engineering, 2018
Integration of intelligent systems in fashion and design related field is a relatively new concep... more Integration of intelligent systems in fashion and design related field is a relatively new concept. The aim of this research was to evaluate the performance of an intelligent fashion styling recommendation system for non-standard female body shapes. As a recommender strategy, the intelligent fashion styling system employs a twostage genetic search, which recommends users fashion styles based on experts’ suggestions and user’s preferences through the system. The proposed intelligent system was tested through a web-based platform using both synthetic and real participants. In cases of synthetic participants results were promising. However, in real participant cases, results were unsatisfactory in average due to certain complications in the testing process and variety in characteristics of participants. Note that, even the average result was unsatisfactory, certain participants indicated their positive remarks for the recommendations provided by intelligent fashion styling platform to identify their expectations.
In this paper, we propose a dedicated application of collective network of binary classifiers (CN... more In this paper, we propose a dedicated application of collective network of binary classifiers (CNBC) to address the problem of incremental learning, which occurs by introducing new SAR terrain classes. Furthermore, another major goal is to achieve a high classification performance over multiple SAR images even though the training data may not be entirely accurate. The CNBC in principle adopts a "Divide and Conquer" type approach by allocating an individual network of binary classifiers (NBCs) to discriminate each SAR terrain class among others and performing evolutionary search to find the optimal binary classifier (BC) in each NBC. Such design further allows dynamic SAR class and feature scalability in such a way that the CNBC can gradually adapt its internal topology to new features and classes with minimal effort. Experiments visually demonstrate the classification accuracy and efficiency of the proposed system over eight fully polarimetric NASA/JPL AIRSAR data sets.
This paper addresses dynamic data clustering as an optimization problem and propose techniques fo... more This paper addresses dynamic data clustering as an optimization problem and propose techniques for finding optimal (number of) clusters in a multi-dimensional data or feature space. In order to accomplish this objective we first propose two novel techniques, which successfully address several major problems in the field of Particle Swarm Optimization (PSO) and promise a significant breakthrough over complex, multi-modal optimization problems at high dimensions. The first one, so-called Multi-Dimensional (MD) PSO, reforms the native structure of swarm particles in such a way that they can make inter-dimensional passes with a dedicated dimensional PSO process. Therefore, in a multidimensional search space where the optimum dimension is unknown, swarm particles can seek both positional and dimensional optima. This eventually removes the necessity of setting a fixed dimension a priori, which is a common drawback for the family of swarm optimizers. Nevertheless, MD PSO is still susceptible to premature convergences due to lack of divergence. To address this problem we propose Fractional Global Best Formation (FGBF) technique, which basically collects all promising dimensional components and fractionally creates an artificial global-best particle (aGB) that has the potential to be a better "guide" than the PSO's native gbest particle. We investigated both individual and mutual applications of the proposed techniques and demonstrated that the best clustering performance can be achieved by their mutual operation. In order to test and evaluate their clustering performance in terms of accuracy, robustness and scalability, a synthetic data-set, which contains ground-truth clusters and offers a broad range of complexity levels is used. An extensive set of experiments demonstrate that the proposed dynamic clustering technique based on MD PSO with FGBF can extract the optimal (number of) clusters by converging to the global optimum of the validity index function at the true dimension.
IEEE Transactions on Biomedical Engineering, May 1, 2022
Objective: Despite the proliferation of numerous deep learning methods proposed for generic ECG c... more Objective: Despite the proliferation of numerous deep learning methods proposed for generic ECG classification and arrhythmia detection, compact systems with the real-time ability and high accuracy for classifying patient-specific ECG are still few. Particularly, the scarcity of patient-specific data poses an ultimate challenge to any classifier. Recently, compact 1D Convolutional Neural Networks (CNNs) have achieved the state-of-the-art performance level for the accurate classification of ventricular and supraventricular ectopic beats. However, several studies have demonstrated the fact that the learning performance of the conventional CNNs is limited because they are homogenous networks with a basic (linear) neuron model. In order to address this deficiency and further boost the patient-specific ECG classification performance, in this study, we propose 1D Self-organized Operational Neural Networks (1D Self-ONNs). Methods: Due to its self-organization capability, Self-ONNs have the utmost advantage and superiority over conventional ONNs where the prior operator search within the operator set library to find the best possible set of operators is entirely avoided. Results: Under AAMI recommendations and with minimal common training data used, over the entire MIT-BIH dataset 1D Self-ONNs have achieved 98% and 99.04% average accuracies, 76.6% and 93.7% average F1 scores on supra-ventricular and ventricular ectopic beat (VEB) classifications, respectively, which is the highest performance level ever reported. Conclusion: As the first study where 1D Self-ONNs are ever proposed for a classification task, our results over the MIT-BIH arrhythmia benchmark database demonstrate that 1D Self-ONNs can surpass 1D CNNs with a significant margin while having a similar computational complexity.
In this paper, we present an automated patient-specific electrocardiogram (ECG) beat classifier d... more In this paper, we present an automated patient-specific electrocardiogram (ECG) beat classifier designed for accurate detection of premature ventricular contractions (PVCs). In the proposed feature extraction scheme, the principal component analysis (PCA) is applied to the dyadic wavelet transform (DWT) of the ECG signal to extract morphological ECG features, which are then combined with the temporal features to form a resultant efficient feature vector. For the classification scheme, we selected the feed-forward artificial neural networks (ANNs) optimally designed by the multi-dimensional particle swarm optimization (MD-PSO) technique, which evolves the structure and weights of the network specifically for each patient. Training data for the ANN classifier include both global (total of 150 representative beats randomly sampled from each class in selected training files) and local (the first 5 min of a patient's ECG recording) training patterns. Simulation results using 40 files in the MIT/BIH arrhythmia database achieved high average accuracy of 97% for differentiating normal, PVC, and other beats.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this p... more The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.
As a consequence of global warming and climate change, the risk and extent of wildfires have been... more As a consequence of global warming and climate change, the risk and extent of wildfires have been increasing in many areas worldwide. Warmer temperatures and drier conditions can cause quickly spreading fires and make them harder to control; therefore, early detection and accurate locating of active fires are crucial in environmental monitoring. Using satellite imagery to monitor and detect active fires has been critical for managing forests and public land. Many traditional statistical-based methods and more recent deep-learning techniques have been proposed for active fire detection. In this study, we propose a novel approach called Operational U-Nets for the improved early detection of active fires. The proposed approach utilizes Self-Organized Operational Neural Network (Self-ONN) layers in a compact U-Net architecture. The preliminary experimental results demonstrate that Operational U-Nets not only achieve superior detection performance but can also significantly reduce computational complexity.
Uploads
Papers by Turker Ince