Activation Functions

Deep learning, specifically the Convolutional Neural Network (CNN), has been a significant technology tool for image processing and human health. CNNs, which mimic the working principles of the human brain, can learn robust representations of images. However, CNNs are susceptible to noise interference, which can impact classification performance. Choosing the right activation function can improve CNNs performance and accuracy. This research aims to test the accuracy of CNN with ResNet50, VGG16, and GoogleNet architectures combined with several activation functions such as ReLU, Leaky ReLU, Sigmoid, and Tanh in the classification of images that experience Poisson noise. Poisson noise is applied to each test data to evaluate CNN accuracy. The data used in this study consists of three scenarios of different numbers of classes, namely 3 classes, 5 classes, and 10 classes. The results showed that combining ResNet50 with the ReLU activation function produced the best performance in class recognition in each scenario of the number of classes experiencing Poisson noise interference. The model achieved 97% accuracy for 3-class data, 95% for 5-class data, and 90% for 10-class data. These results show that using ResNet50 with the ReLU activation function can provide excellent resistance to Poisson noise in image processing. It was found that as the number of classes increases, the accuracy of image recognition tends to decrease. This shows that the more complex the image classification task is with a larger number of classes, the more difficult it is for CNNs to distinguish between different classes.

Since statistical analysis of poetry is a challenging task in Natural Language Processing (NLP), making inferences about the poets also becomes a very challenging task. In this study, a dataset of Turkish poems which is obtained for 5 different poets is used to compare classification performance of the Artificial Neural Network (ANN) and Deep Neural Network (DNN) architectures. While Multilayer Perceptron (MLP) is selected for ANN architecture, Convolutional Neural Network (CNN) is selected as DNN architecture. Two main feature representation approaches are used for the experiments-Term Frequency-Inverse Document Frequency (TF-IDF) is used for ANN and word embedding is used for DNN. As a result of the experiments it has been seen that MLP has the highest performance in terms of accuracy, precision, recall and F-score.

The traditional and classical ReLU activation function (AF) has been extensively applied in deep neural networks, in particular Convolutional Neural Networks (CNN), for image classification despite its unresolved 'dying ReLU problem', which poses challenges to reliable applications of CNN in both academic and industry research and development. This issue leading to misclassifications and lack of generalisation has obvious important implications for critical applications, such as those in healthcare. Recent approaches are still stepping in a similar direction by just proposing variations of the activation function while maintaining the solution within the same unresolved 'dying ReLU' challenge. This contribution reports a different research direction by investigating the development of an innovative quantum approach to the ReLU AF that avoids the 'dying ReLU problem' by disruptive design. The proposed Leaky ReLU approach was leveraged as a baseline ReLU function on which the two quantum principles of entanglement and superposition were applied to derive the proposed Quantum ReLU (QReLU) and the modified-QReLU (m-QReLU) activation functions. Both QReLU and m-QReLU are implemented and made freely available in TensorFlow and Keras. This original approach is effective and validated extensively in case studies that facilitate the detection of COVID-19 and Parkinson's Disease (PD) from medical images. The two novel AFs were evaluated in a two-layered convolutional neural network (CNN) against nine ReLU-based AFs on seven benchmark datasets. The MNIST dataset was used for an initial validation. Thereafter, the CNN with the two proposed Quantum Activation Functions was evaluated on four benchmark datasets on images of spiral drawings taken via graphic tablets from patients with Parkinson's Disease and healthy subjects, as well as two benchmark datasets on point-of-care ultrasound images on the lungs of patients with COVID-19, those with pneumonia and healthy controls. Despite a higher computational cost, results indicated an overall higher classification accuracy, precision, recall and F1-score brought about by either quantum AFs on five of the seven benchmark datasets, thus demonstrating its potential to be the new benchmark or gold standard AF in CNNs and aid image classification tasks involved in critical applications, such as medical diagnoses of COVID-19 and PD.

Dense direct RGB-D registration methods are widely used in tasks ranging from localization and tracking to 3D scene reconstruction. This work addresses a peculiar aspect which drastically limits the applicability of direct registration, namely the weakness of the convergence domain. First, we propose an activation function based on the conditioning of the RGB and ICP point-to-plane error terms. This function strengthens the geometric error influence in the first coarse iterations, while the intensity data term dominates in the finer increments. The information gathered from the geometric and photometric cost functions is not only considered for improving the system observability, but for exploiting the different convergence properties and convexity of each data term. Next, we develop a set of strategies as a flexible regularization and a pixel saliency selection to further improve the quality and robustness of this approach. The methodology is formulated for a generic warping model and results are given using perspective and spherical sensor models. Finally, our method is validated in different RGB-D spherical datasets, including both indoor and outdoor real sequences and using the KITTI VO/SLAM benchmark dataset. We show that the different proposed techniques (weighted activation function, regularization, saliency pixel selection), lead to faster convergence and larger convergence domains, which are the main limitations to the use of direct methods.

The development of Artificial Neural Networks (ANNs) has achieved a lot of fruitful results so far, and we know that activation function is one of the principal factors which will affect the performance of the networks. In this work, the role of many different types of activation functions, as well as their respective advantages and disadvantages and applicable fields are discussed, so people can choose the appropriate activation functions to get the superior performance of ANNs.

The increase in population has led to increase in demand of automobiles across the globe. Concerns about the pollutants emitted from an engine are growing periodically. The paper has tried to show an exploratory review of the various methodologies adopted for accurately measuring and analysing exhaust engine emissions. The paper has the objective of showing the main conclusions of the recent research performed since last decade (2008 to 2020) for the above mentioned topic with the help of artificial intelligence methodologies. This review addresses an important application of artificial intelligence that can be applied in measuring the engine emissions. The measurement of these engine emissions are mandatory for every automobile company. This task is usually achieved by repeated testing of automobile which is not a cost efficient process. These process usually involves expensive test rigs installation. But predictive modeling for accurate testing of emissions can be used as a digital/virtual tool. Hence there is an extensive scope of research in this field. The paper has tried to showcase emerging trend of advanced machine learning algorithms that can easily help in generating emission data in a simple manner.

The experimental investigation on the efficient learning of highly non-linear problems by online training, using ordinary feed forward neural networks and stochastic gradient descent on the errors computed by back-propagation, gives evidence that the most crucial factors for efficient training are the hidden units' differentiation, the attenuation of the hidden units' interference and the selective attention on the parts of the problems where the approximation error remains high. In this report, we present global and local selective attention techniques and a new hybrid activation function that enables the hidden units to acquire individual receptive fields which may be global or local depending on the problem's local complexities. The presented techniques enable very efficient training on complex classification problems with embedded subproblems.

Parkinson's Disease (PD) automatic identification in early stages is one of the most challenging medicine-related tasks to date, since a patient may have a similar behaviour to that of a healthy individual at the very early stage of the disease. In this work, we cope with PD automatic identification by means of a Convolutional Neural Network (CNN), which aims at learning features from a signal extracted during the individual's exam by means of a smart pen composed of a series of sensors that can extract information from handwritten dynamics. We have shown CNNs are able to learn relevant information, thus outperforming results obtained from raw data. Also, this work aimed at building a public dataset to be used by researchers worldwide in order to foster PD-related research.

COVID-19 has become a pandemic and is a big problem that needs to be checked out immediately. CT scan images can explain the lung conditions of COVID-19 patients and have the potential to be a clinical diagnostic tool. In this research, we classify COVID-19 by recognizing images on a computer tomography scan (CT scan) of the lungs using digital image processing and GLCM feature extraction techniques to obtain grayscale level values in CT images, followed by the creation of an artificial neural network model. So that the model can classify CT scan images, the results in this research obtained the most optimal model for COVID-19 classification performance with 90% accuracy, 88% precision, 91% recall, and 90% F1 score. This research can be a useful tool for clinical practitioners and radiologists to assist them in the diagnosis, quantification, and follow-up of COVID-19 cases.

Regularly inflected forms often behave differently in language production than irregular forms. These differences are often used to argue that irregular forms are listed in the lexicon but regular forms are produced by rule. Using an experimental speech production task with adults, it is shown that overtensing errors, where a tensed verb is used in place of an infinitive, predominantly involve irregular forms, but that the differences may be due to phonological confounds, not to regularity per se. Errors involve vowel-changing irregular forms more than suffixing inflected forms, with at best a small difference between regular-ed and irregular-en. Frequency effects on overtensing errors require a model in which the past-tense and base forms of the verb are in competition and in which activation functions are nonlinear, and rule out models with specialized subnetworks for past-tense forms. Implications for theories of language production are discussed.

Predicting electricity power is an important task, which helps power utilities in improving their systems’ performance in terms of effectiveness, productivity, management and control. Several researches had introduced this task using three main models: engineering, statistical and artificial intelligence. Based on the experiments, which used artificial intelligence models, multilayer neural networks model has proven its success in predicting many evaluation datasets. However, the performance of this model depends mainly on the type of activation function. Therefore, this paper introduces an experimental study for investigating the performance of the multilayer neural networks model with respect to different activation functions and different depths of hidden layers. The experiments in this paper cover the comparison among eleven activation functions using four benchmark electricity datasets. The activation functions under examination are sigmoid, hyperbolic tangent, SoftSign, SoftPl...

The increase in population has led to increase in demand of automobiles across the globe. Concerns about the pollutants emitted from an engine are growing periodically. The paper has tried to show an exploratory review of the various methodologies adopted for accurately measuring and analysing exhaust engine emissions. The paper has the objective of showing the main conclusions of the recent research performed since last decade (2008 to 2020) for the above mentioned topic with the help of artificial intelligence methodologies. This review addresses an important application of artificial intelligence that can be applied in measuring the engine emissions. The measurement of these engine emissions are mandatory for every automobile company. This task is usually achieved by repeated testing of automobile which is not a cost efficient process. These process usually involves expensive test rigs installation. But predictive modeling for accurate testing of emissions can be used as a digital/virtual tool. Hence there is an extensive scope of research in this field. The paper has tried to showcase emerging trend of advanced machine learning algorithms that can easily help in generating emission data in a simple manner.

Object detection is a subfield of computer vision that is currently heavily based on machine learning. For the past decade, the field of machine learning has been dominated by so-called deep neural networks, which take advantage of improvements in computing power and data availability. A subtype of a neural network called a convolutional neural network (CNN) is well-suited for image-related tasks. The network is trained to look for different features, such as edges, corners and color differences, across the image and to combine these into more complex shapes. For project, we reviewed the current literature on convolutional object detection and tested the implement ability of one of the methods. We found that convolutional object detection is still evolving as a technology, despite outranking other object detection methods. By virtue of free availability of datasets and pre-trained networks, it is possible to create a functional implementation of a deep neural network without access to specialist hardware. Pretrained networks can also be used as a starting point for training new networks, decreasing costly training time.

Artificial neural networks simulate the neural systems behaviour by means of the interconnection of the basic processing units called neurons. The neurons canreceive external signals or signals coming from the otherneurons affected by a factor called weight. The output ofneuron is the result of applying a specific function, knownas transfer function, to the sum of its inputs plus thresholdvalue called bias. This paper demonstrates the practical analysis of neural network back-propagation algorithm.It shows the mathematical process of how the neural network manages the data fed to it for it to be trained to recognize patterns, classify data and forecast future events. Feed forward networks have been employed along with back propagation algorithm for the pattern recognition process.

Predicting electricity power is an important task, which helps power utilities in improving their systems’ performance in terms of effectiveness, productivity, management and control. Several researches had introduced this task using three main models: engineering, statistical and artificial intelligence. Based on the experiments, which used artificial intelligence models, multilayer neural networks model has proven its success in predicting many evaluation datasets. However, the performance of this model depends mainly on the type of activation function. Therefore, this paper introduces an experimental study for investigating the performance of the multilayer neural networks model with respect to different activation functions and different depths of hidden layers. The experiments in this paper cover the comparison among eleven activation functions using four benchmark electricity datasets. The activation functions under examination are sigmoid, hyperbolic tangent, SoftSign, SoftPl...

Nowadays the entire world depends on emails as a communication tool. Spammers try to exploit various vulnerabilities to attack users with spam emails. While it is difficult to prevent spam email attacks, many research studies have been developed in the last decade in an attempt to detect spam emails. These studies were conducted using machine learning techniques and various types of neural networks. However, with all their attempts the highest accuracy acquired was 94.2% by random forest classifier. Deep learning techniques have demonstrated higher accuracy performance compared to the traditional machine learning algorithms. In this paper, deep recurrent neural network was used to determine whether an email is a spam email. After investigating different configurations for this method, the best setting that generated the highest accuracy was based on using Tanh as the activation function with the dropout rate equals to 0.1 and the number of epochs achieving 100. The proposed approach...

Application of machine learning in multiclass classification of brain tumor types has contributed to the development of computer aided diagnosis (CAD) system that can potentially enhance accuracy and speed up diagnosis of the disease. LDA+ELM model with different activation functions were investigated to achieve the optimum performances in terms of accuracy, Kappa statistic, sensitivity, precision, F-measure, training time and test time.  We also proposed a user-friendly GUI in characterizing brain tumor types using MR images. First, a total of 3064 slices of CE T1-weighted brain MR images with ground truth were downloaded from a free online database. The manually segmented tumor region was augmented and then undergo several feature extraction techniques. All the feature descriptors obtained were then concatenated, followed by LDA dimensionality approach. Performance of different number of LDA features and ELM activation functions were investigated by repeated training and test. The...

The traditional and classical ReLU activation function (AF) has been extensively applied in deep neural networks, in particular Convolutional Neural Networks (CNN), for image classification despite its unresolved 'dying ReLU problem', which poses challenges to reliable applications of CNN in both academic and industry research and development. This issue leading to misclassifications and lack of generalisation has obvious important implications for critical applications, such as those in healthcare. Recent approaches are still stepping in a similar direction by just proposing variations of the activation function while maintaining the solution within the same unresolved 'dying ReLU' challenge. This contribution reports a different research direction by investigating the development of an innovative quantum approach to the ReLU AF that avoids the 'dying ReLU problem' by disruptive design. The proposed Leaky ReLU approach was leveraged as a baseline ReLU function on which the two quantum principles of entanglement and superposition were applied to derive the proposed Quantum ReLU (QReLU) and the modified-QReLU (m-QReLU) activation functions. Both QReLU and m-QReLU are implemented and made freely available in TensorFlow and Keras. This original approach is effective and validated extensively in case studies that facilitate the detection of COVID-19 and Parkinson's Disease (PD) from medical images. The two novel AFs were evaluated in a two-layered convolutional neural network (CNN) against nine ReLU-based AFs on seven benchmark datasets. The MNIST dataset was used for an initial validation. Thereafter, the CNN with the two proposed Quantum Activation Functions was evaluated on four benchmark datasets on images of spiral drawings taken via graphic tablets from patients with Parkinson's Disease and healthy subjects, as well as two benchmark datasets on point-of-care ultrasound images on the lungs of patients with COVID-19, those with pneumonia and healthy controls. Despite a higher computational cost, results indicated an overall higher classification accuracy, precision, recall and F1-score brought about by either quantum AFs on five of the seven benchmark datasets, thus demonstrating its potential to be the new benchmark or gold standard AF in CNNs and aid image classification tasks involved in critical applications, such as medical diagnoses of COVID-19 and PD.

An autonomous mobile robot is a robot which can move and act autonomously without the help of human assistance. Navigation problem of mobile robot in unknown environment is an interesting research area. This is a problem of deducing a path for the robot from its initial position to a given goal position without collision with the obstacles. Different methods such as fuzzy logic, neural networks etc. are used to find collision free path for mobile robot. This paper examines behavior of path planning of mobile robot using three activation functions of wavelet neural network i.e. Mexican Hat, Gaussian and Morlet wavelet functions by MATLAB. The simulation result shows that WNN has faster learning speed with respect to traditional artificial neural network.

Activation functions facilitate deep neural networks by introducing non-linearity to the learning process. The non-linearity feature gives the neural network the ability to learn complex patterns. Recently, the most widely used activation function is the Rectified Linear Unit (ReLU). Though, other various existing activation including hand-designed alternatives to ReLU have been proposed. However, none has succeeded in replacing ReLU due to their existing inconsistencies. In this work, activation function called ReLU-Memristor-like Activation Function (RMAF) is proposed to leverage benefits of negative values in neural networks. RMAF introduces a constant parameter (α) and a threshold parameter (p) making the function smooth, non-monotonous, and introduces non-linearity in the network. Our experiments show that, the RMAF works better than ReLU and other activation functions on deeper models and across number of challenging datasets. Firstly, experiments are performed by training and classifying on multi-layer perceptron (MLP) over benchmark data such as the Wisconsin breast cancer, MNIST, Iris and Car evaluation. RMAF achieves high performance of 98.74%, 99.67%, 98.81% and 99.42% respectively, compared to Sigmoid, Tanh and ReLU. Secondly, experiments were performed on convolution neural network (ResNet) over MNIST, CIFAR-10 and CIFAR-100 data and observed the proposed activation function achieves higher performance accuracy of 99.73%, 98.77% and 79.82% respectively than Tanh, ReLU and Swish. Additionally, we experimented our work on deep networks i.e. squeeze network (SqueezeNet), Dense connected neural network (DenseNet121) and ImageNet dataset, which RMAF produced the best performance. We note that, the RMAF converges faster than the other functions and can replace ReLU in any neural network due to the efficiency, scalability and its similarity to both ReLU and Swish.

This paper describes an application of a pre-trained Deep Convolutional Neural Network to detect meteors from night sky images. The dataset is relatively small, composed of labeled images of meteors and non-meteors from the night sky. Techniques like data augmentation were used to create data artificially, and a dropout layer was introduced to prevent overfitting at the training augmented dataset. The performance obtained by the methods using traditional five-fold cross-validation along with changing of the images for black and white, compared with the ones obtained using only the training and validation partitions and RGB colors, achieved a higher mean accuracy of 84.35%. Resumo: Este artigo descreve a aplicação de uma rede neural convolucional profunda prétreinada para detectar meteoros a partir de imagens do céu noturno. O conjunto de dadosé relativamente pequeno, composto de imagens rotuladas de meteoros e não-meteoros. Técnicas como aumento de dados foram usadas para criar dados artificialmente, e uma camada de dropout foi introduzida para evitar overfitting no dataset aumentado para treinamento. O desempenho obtido pelos métodos usando a validação cruzada com cinco partições, juntamente com a mudança das imagens para preto e branco, em comparação com os obtidos usando apenas as partições de treinamento e validação e cores RGB, alcançou uma precisão média maior de 84,35 %

Predicting electricity power is an important task, which helps power utilities in improving their systems’ performance in terms of effectiveness, productivity, management and control. Several researches had introduced this task using three main models: engineering, statistical and artificial intelligence. Based on the experiments, which used artificial intelligence models, multilayer neural networks model has proven its success in predicting many evaluation datasets. However, the performance of this model depends mainly on the type of activation function. Therefore, this paper introduces an experimental study for investigating the performance of the multilayer neural networks model with respect to different activation functions and different depths of hidden layers. The experiments in this paper cover the comparison among eleven activation functions using four benchmark electricity datasets. The activation functions under examination are sigmoid, hyperbolic tangent, SoftSign, SoftPl...

Cataract are the highest cause of blindness that there are 32.4 million people experiencing blindness and as many as 191 million people experiencing visual disabilities in 2010 in the world. On the other hand, the longer a patient suffers from cataracts or late treatment. The development of cataract identification using a traditional algorithm based on feature representation is highly dependent on the classification process carried out by an eye specialist so that the method is prone to misclassification of a person detected or not. However, at this time there is a deep learning, convolutional neural network (CNN) which is used for pattern recognition which can help automate image classification. This research was conducted to increase the accuracy value and minimize data loss in the process of cataract identification by performing an experience namely the manipulation process was carried out by changing epochs. The results of this study indicate that the addition of epochs affects ...

This paper presents overview of applications of artificial neural networks (ANN) in the field of engine development. Various approaches using ANN are highlighted that resulted in better modeling of engine operations. Using ANN we can reduce engine development time. The paper discusses ANN approach, algorithms and importance of architecture. This will also help in advancing ANN research.

The experimental investigation on the efficient learning of highly non-linear problems by online training, using ordinary feed forward neural networks and stochastic gradient descent on the errors computed by back-propagation, gives evidence that the most crucial factors for efficient training are the hidden units' differentiation, the attenuation of the hidden units' interference and the selective attention on the parts of the problems where the approximation error remains high. In this report, we present global and local selective attention techniques and a new hybrid activation function that enables the hidden units to acquire individual receptive fields which may be global or local depending on the problem's local complexities. The presented techniques enable very efficient training on complex classification problems with embedded subproblems.

The choice of transfer functions may strongly influence complexity and performance of neural networks used in classification and approximation tasks. A taxonomy of activation and output functions is proposed, allowing to generate many transfer functions. Several less-known types of transfer functions and new combinations of activation/output functions are described. Functions parameterize to change from localized to delocalized type, functions with activation based on nonEuclidean distance measures, bicentral functions formed from pairs of sigmoids are discussed

This paper presents overview of applications of artificial neural networks (ANN) in the field of engine development. Various approaches using ANN are highlighted that resulted in better modeling of engine operations. Using ANN we can reduce engine development time. The paper discusses ANN approach, algorithms and importance of architecture. This will also help in advancing ANN research.

An image processing system that based computer vision has received many attentions from science and technology expert. Research on image processing is needed in the development of human-computer interactions such as hand recognition or gesture recognition for people with hearing impairments and deaf people. In this research we try to collect the hand gesture data and used a simple deep neural network architecture that we called model E to recognize the actual hand gestured. The dataset that we used is collected from kaggle.com and in the form of ASL (American Sign Language) datasets. We doing accuracy comparison with another existing model such as AlexNet to see how robust our model. We find that by adjusting kernel size and number of epoch for each model also give a different result. After comparing with AlexNet model we find that our model E is perform better with 96.82% accuracy.

This paper presents overview of applications of artificial neural networks (ANN) in the field of engine development. Various approaches using ANN are highlighted that resulted in better modeling of engine operations. Using ANN we can reduce engine development time. The paper discusses ANN approach, algorithms and importance of architecture. This will also help in advancing ANN research.

Activation functions facilitate deep neural networks by introducing non-linearity to the learning process. The non-linearity feature gives the neural network the ability to learn complex patterns. Recently, the most widely used activation function is the Rectified Linear Unit (ReLU). Though, other various existing activation including hand-designed alternatives to ReLU have been proposed. However, none has succeeded in replacing ReLU due to their existing inconsistencies. In this work, activation function called ReLU-Memristor-like Activation Function (RMAF) is proposed to leverage benefits of negative values in neural networks. RMAF introduces a constant parameter (α) and a threshold parameter (p) making the function smooth, non-monotonous, and introduces non-linearity in the network. Our experiments show that, the RMAF works better than ReLU and other activation functions on deeper models and across number of challenging datasets. Firstly, experiments are performed by training and classifying on multi-layer perceptron (MLP) over benchmark data such as the Wisconsin breast cancer, MNIST, Iris and Car evaluation. RMAF achieves high performance of 98.74%, 99.67%, 98.81% and 99.42% respectively, compared to Sigmoid, Tanh and ReLU. Secondly, experiments were performed on convolution neural network (ResNet) over MNIST, CIFAR-10 and CIFAR-100 data and observed the proposed activation function achieves higher performance accuracy of 99.73%, 98.77% and 79.82% respectively than Tanh, ReLU and Swish. Additionally, we experimented our work on deep networks i.e. squeeze network (SqueezeNet), Dense connected neural network (DenseNet121) and ImageNet dataset, which RMAF produced the best performance. We note that, the RMAF converges faster than the other functions and can replace ReLU in any neural network due to the efficiency, scalability and its similarity to both ReLU and Swish.

Deep Learning technologies show promise for dramatic advances in fields such as image classification and speech recognition. Deep Learning (DL) is a class of Machine Learning algorithms that involves learning of multiple levels of features from data to build a model. One of the open questions in DL is whether up-to-date models can be built and provide dealing with dynamic and large volumes of new data created. This requires addressing how models can be consistently constructed and updated (incremental learning) in a scalable manner. Current research and practices of DL do not fully support these important features, such as distributed learning or incremental learning to an extent that is required. The objective of this thesis is to provide a solution to this problem by building a framework that is distributed and incremental in nature. In the DL-DI framework, a learning problem is composed of two stages: Local Learning and Global Learning. In the local learning stage, a learning pro...

Abstract. The choice of transfer functions may strongly influence complexity and performance of neural networks used in classification and approximation tasks. A taxonomy of activation and output functions is proposed, allowing to generate many transfer functions. Several less-...

Abstract. The choice of transfer functions may strongly influence complexity and performance of neural networks used in classification and approximation tasks. A taxonomy of activation and output functions is proposed, allowing to generate many transfer functions. Several less-...

Image classification is a complex computer vision problem. Recently, there have been endeavors using convolutional and deep neural network architectures (CNN- DNN). There are very deep neural networks architectures available for image classification. These networks are taking days and even weeks to learn the network. It will be time consuming if it is to be tested in various ways to find an efficient architecture. It gives rise to shallow neural networks which are having few layers. Due to this fact, such types of neural network architectures have been implemented in this research endeavor. Recently, several other improvements have been made on CNNs by way of introducing several types of convolutions and dif erent types of activation functions. The shallow network makes it possible to test these dif erent types of networks and finding the efficient network architecture given the less amount of time to learn in such type of network architectures. In this research five dif erent CNN-DNN architectures have been analyzed on an image set having fifteen categories of images and their results were compared. The architectures were varied by image sizes taken, dif erent number of filters, dif erent sub-sampling processes and dif erent activation functions. It is found that an architecture with a combination of Relu and Hyperbolic activation functions were performing well over other architectures and other combinations of activation functions.

Image classification is a complex computer vision problem. Recently, there have been endeavors using convolutional and deep neural network architectures (CNN- DNN). There are very deep neural networks architectures available for image classification. These networks are taking days and even weeks to learn the network. It will be time consuming if it is to be tested in various ways to find an efficient architecture. It gives rise to shallow neural networks which are having few layers. Due to this fact, such types of neural network architectures have been implemented in this research endeavor. Recently, several other improvements have been made on CNNs by way of introducing several types of convolutions and dif erent types of activation functions. The shallow network makes it possible to test these dif erent types of networks and finding the efficient network architecture given the less amount of time to learn in such type of network architectures. In this research five dif erent CNN-DNN architectures have been analyzed on an image set having fifteen categories of images and their results were compared. The architectures were varied by image sizes taken, dif erent number of filters, dif erent sub-sampling processes and dif erent activation functions. It is found that an architecture with a combination of Relu and Hyperbolic activation functions were performing well over other architectures and other combinations of activation functions.

Nowadays, monitoring using a surveillance camera is the main challenging task. In this regard, human detection in video sequences is one of the challenging approaches. In a variety of real-world applications, human identification in videos (i.e., a sequence of images) is critical (e.g., visual surveillance and automated driver assistance). In this work, we are using a sliding window algorithm to detect a human in the video frame sequences. Hence false positive is another challenging task. Extensive work is carried out to reduce the false positive of this work to make it applicable in a real-world scenario.

Image classification is a complex computer vision problem. Recently, there have been endeavors using convolutional and deep neural network architectures (CNNDNN). There are very deep neural networks architectures available for image classification. These networks are taking days and even weeks to learn the network. It will be time consuming if it is to be tested in various ways to find an efficient architecture. It gives rise to shallow neural networks which are having few layers. Due to this fact, such types of neural network architectures have been implemented in this research endeavor. Recently, several other improvements have been made on CNNs by way of introducing several types of convolutions and different types of activation functions. The shallow network makes it possible to test these different types of networks and finding the efficient network architecture given the less amount of time to learn in such type of network architectures. In this research five different CNN-DNN architectures have been analyzed on an image set having fifteen categories of images and their results were compared. The architectures were varied by image sizes taken, different number of filters, different sub-sampling processes and different activation functions. It is found that an architecture with a combination of Relu and Hyperbolic activation functions were performing well over other architectures and other combinations of activation functions.

Image classification is a complex computer vision problem. Recently, there have been endeavors using convolutional and deep neural network architectures (CNN-DNN). There are very deep neural networks architectures available for image classification. These networks are taking days and even weeks to learn the network. It will be time consuming if it is to be tested in various ways to find an efficient architecture. It gives rise to shallow neural networks which are having few layers. Due to this fact, such types of neural network architectures have been implemented in this research endeavor. Recently, several other improvements have been made on CNNs by way of introducing several types of convolutions and different types of activation functions. The shallow network makes it possible to test these different types of networks and finding the efficient network architecture given the less amount of time to learn in such type of network architectures. In this research five different CNN-DNN architectures have been analyzed on an image set having fifteen categories of images and their results were compared. The architectures were varied by image sizes taken, different number of filters, different sub-sampling processes and different activation functions. It is found that an architecture with a combination of Relu and Hyperbolic activation functions were performing well over other architectures and other combinations of activation functions.

In this project we have implemented facial feature extraction and detection to detect 7 categories of emotions in a person, namely 'happy', 'sad', 'surprise', 'anger', 'fear', 'disgust' and 'stress'. This is done by special type of Deep Learning technique known as Convolution Neural Network (CNN) framework to detect the emotions. We report accuracies of 91.8% in the FER-2013 dataset. These emotions are checked by using real time webcam emotion feed, which checks emotion frame-by-frame. Since age and gender also play a significant role in our general day-today social interactions hence, we have tried to bridge the gap between age and gender estimation methods and real time face recognition through deep convolution networks. We report accuracies of 93% in gender classification and about 50% in age classification though the IMDb-Wikipedia dataset. Further, we have developed a machine learning model to detect the emotions through speech. Accuracy of 98 % has been reported for speech classification using RAVDESS and SAVEE datasets. Along with this we have prepared a self-assessment test for emotion detection and later on for mental health being.We prepared a test for emotion detection in which percentage of dominating emotion among: ‘happy’, ‘sad’, ‘depression’, ‘anxiety’, ‘fear’ and ‘anger’ is calculated. Based on the score obtained, the person is asked to attend a Computer Adaptive test (CAT) to check percentage of ‘anger’, ‘depression’ or ‘anxiety’, and a proper solution to improve their mental health is provided according to the score they got.