Features Fusion

In modern agriculture, ensuring plant health is essential for high crop yields and quality. Plant diseases pose risks to economies, communities, and the environment, making early and accurate diagnosis crucial. The internet of things (IoT) has revolutionized farming by enabling real-time crop monitoring and using drones and cameras for early disease detection. This technology helps farmers address challenges with precision and sustainability. This research proposes an ensemble learning model incorporating multi-class capsule networks (MCCN) and other pre-trained model with majority voting system is implemented to predict plant diseases and pests early. The research aims to develop a robust MCCN-based ensemble prediction model for timely disease identification. To evaluate the performance of the ensemble model, various key metrics, including accuracy, and loss value, are assessed. Furthermore, a comparative analysis is conducted, benchmarking the MCCN model against other well-known pre-trained models such as residual network-101 (ResNet101), visual geometry group-19 (VGG19), and GoogleNet. This research signifies a substantial stride towards the realization of IoT-driven precision agriculture, where advanced technology and machine learning contribute to the early detection and mitigation of plant diseases, ultimately enhancing crop yield and environmental sustainability.

An essential component of maintaining global food production is plants. On other hand, a number of plant diseases can threaten agricultural output and cause large losses if left unchecked. Agricultural specialists and botanists physically track plant diseases in a labor-intensive, error-prone manner using a conventional method. AI can give evaluations that are quicker and more accurate than those made using conventional approaches by automating the identification and analysis of diseases. This technical development presents a viable way to lessen crop losses and lessen the severity of infections. As a result, we describe an ensemble machine learning strategy for plant disease classification in this study that is enabled by deep learning. Data augmentation is done in the first part of the study, and in the second step, we provide a modified mask region-based convolutional neural network (R-CNN) model for plant leaf segmentation. Afterwards, a model to extract the deep features based on CNN is shown. Lastly, the ensemble classifier is built using support vector machine classifier (SVM), random forest (RF), and decision tree (DT) with the aid of majority voting. The suggested method's effectiveness is tested on plant village, apple, maize, and rice, yielding overall accuracy values of 99.45%, 96.30%, 96.85%, and 98.25%, in that order.

SDP is the term that describes the most significant step in the process of identifying fault-prone components provided in the case of the software development life cycle. Its main agenda is not only to enhance software quality but also to cut down maintenance cost in the long run. Although the sheer amount of research dedicated to SDP is quite high, the current models still have numerous limitations in its practice such as high false positive rates and the serious imbalance of the number of defective and substitute modules. Natural limitations negatively affect the learning capabilities of predictive models and lower the accuracy and quality of the debugging process as a whole. To solve these nagging issues, the current study has proposed a robust and strong SDP model that builds on a mixture of sophisticated computing techniques, in particular, those that are eventualities of making precise forecasts and model generalization. The method stated here starts with a thorough preprocessing bezel on the information, during which the information will be normalized in terms of features, so that all elements are on a similar scale. The move serves to reduce the impact of outliers and noisy data hence improving the quality of training dataset. After preprocessing, a class imbalance will be solved with the help of Minority Oversampling by Synthetic Data (MOSD) technique. This can be done by creating synthetic data of the minority class to get a more balanced distribution of defective and good examples which is essential toward good training of classifiers. After that, an Adaptive Sequential K-Best (ASKB) feature selection algorithm is applied to highlight the most considerable and informative features. This approach analytically discerns the significance of any attribute in a dynamic fashion; thus decreasing the dimension of the dataset with negligible loss of crucial information on predictive variables. The slimmed down feature set will also help in building a model that can be interpreted better and is also computationally economical. As a classification algorithm a Weighted Random Forest (WRF) is used in the case of the classification task. This extension of the conventional Random Forest incorporates instance-based weighting where the model is made to give more weight to developing accurate labels of instances of the minority class. The WRF classifier, in turn, improves the overall performance in prediction and lessens the bias in relation to the dominant category. The empirical assessment of the offered structure proves its capabilities to be highly advanced in comparison to the existing models. The method produced outstanding classification accuracy measures that were 99.11999, 99.43111, 99.12199 and 99.33333 in terms of accuracy rate, precision, recall and F1-score respectively. The outcomes can substantiate the usefulness of the combined methodology on enhancing defect forecasting abilities. Moreover, the suggested model has practical implications that can be employed in proactive software quality assurance, thus more reliable and cost-effective software engineering processes can be conducted.

In this paper, we present a machine vision system that was developed on the basis of neural networks to identify twelve houseplants. Image processing system was used to extract 41 features of color, texture and shape from the images taken from front and back of the leaves. The features were fed into the neural network system as the recognition criteria and inputs. Multilayer perceptron (MLP) neural network with Declining Learning-Rate Factor algorithm (BDLRF) training algorithm was used as a classifier. Classification was done in three stages based on eligibility and strength of characteristics in identifying the plants. Eligibility criteria were assessed at each stage using plants class resolution power. In this classification method, each step requires a small number of attributes and for this reason its speed and accuracy can be very high. The results showed that the accuracy of classification of plants in three steps reaches 100%. Also, the optimal features for classification included three inputting steps of morphological features, HSI color features extracted from back of the leaves, and HSI texture features of the back of the leaves.

India, among the agriculture-based economy grows wide variety of rice along with other crops. These varieties have different commercial values as they are different in their features. It becomes extremely challenging to classify rice varieties which have similar features but are different in their quality. This study considers four varieties of similar looking rice which conform to be Sona-Masuri. A total of 4180 images are considered to extract 56 features including textural, red, green, blue (RGB) and hue, saturation, value (HSV) color and wavelet decomposition. Machine learning (ML) models like support vector machine (SVM), K-nearest neighbor (KNN), decision tree (DT) and voting classifiers are developed for feature dataset and convolutional neural network (CNN) model for image dataset. The results obtained for every model are obtained using statistical methods and the results are expressed in a table for accuracy, precision, recall and F1-score. A classification accuracy of 72.48% is obtained for SVM using polynomial kernel trick by considering all 56 features. The customized CNN model is designed with three convolution layers has resulted in 97.13% of training accuracy and 87.5% of validation accuracy. Based on the results obtained, it is witnessed that the ML models employed in this study to classify rice types with similar appearances have practical applications.

This chapter proposes a novel hybrid approach that combines the strengths of wavelet transform with the powerful learning capabilities of autoencoder networks to achieve superior denoising performance. By leveraging wavelet decomposition to process images at multiple scales and feeding these decomposed signals into a deep autoencoder network, we effectively suppress noise while maintaining high-frequency details. Extensive experiments demonstrate that our method outperforms existing techniques, yielding significant improvements in both peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). The results suggest that the integration of wavelet transform and autoencoder networks offers a promising solution for robust image denoising, especially in scenarios with complex noise patterns.

Effective crop categorization is important for keeping track of how crops grow and how much they produce in the future. Gathering crop data on categories, regions, and space distribution in a timely and accurate way could give a scientifically sound reason for changes to the way crops are organized. Polarimetric synthetic aperture radar dataset provides sufficient information for accurate crop categorization. It is essential to classify crops in order to successfully. This article presents wavelet transform (WT) based optimizedlong short-term memory (LSTM) deep learning (DL) for effective crop categorization. Image denoising is performed by WT. Denoising algorithms for images attempt to find a middle ground between totally removing all of the image's noise and preserving essential, signal-free components of the picture in their original state. After denoising of images, crop image classification is achieved by LSTM and support vector machine (SVM) algorithm. LSTM has achieved 99.5% accuracy.

This investigation presents a groundbreaking deep learning approach for the
early recognition and differentiation of Alzheimer's Disease (AD), a
widespread neurological disorder, employing non-invasive cerebral imaging
techniques. By combining state-of-the-art deep learning algorithms with
feature fusion methodologies, the research seeks to boost the accuracy and
consistency of AD diagnosis, potentially facilitating earlier intervention and
individualized treatment approaches. The framework utilizes multiple imaging
modalities, such as Magnetic Resonance Imaging (MRI) and Positron
Emission Tomography (PET), to extract complementary features that reflect
both structural and functional brain alterations associated with AD. An
advanced feature fusion strategy is implemented to integrate these
heterogeneous data sources into a unified representation, allowing the deep
learning model to discern more intricate and distinctive patterns of
neurodegeneration. The deep neural network, trained on an extensive,
clinically diverse dataset, is engineered to identify subtle AD biomarkers,
even in preclinical phases, offering a robust tool for healthcare professionals.
By concentrating on non-invasive brain imaging techniques, the proposed
methodology presents a promising alternative to traditional diagnostic
procedures, which often involve invasive or time-consuming examinations.
The integration of clinical expertise and sophisticated machine learning in
this framework provides a powerful resource for neurological specialists,
enhancing diagnostic precision and supporting more informed therapeutic
decisions. Ultimately, this study showcases the potential of deep learning and
feature fusion techniques to revolutionize the clinical approach to AD
diagnosis, marking a significant advancement toward more efficient and
accessible healthcare solutions.

Context: Just-in-time defect prediction (JIT-DP) is a crucial process in software development that focuses on identifying potential defects during code changes, facilitating early mitigation and quality assurance. Pretrained language models like CodeBERT have shown promise in various applications but often struggle to distinguish between defective and non-defective code, especially when dealing with noisy labels.
Objective: The primary aim of this study is to enhance the robustness of pre-trained language models in identifying software defects by developing an innovative framework that leverages contrastive learning and feature fusion.
Method: We introduce JIT-CF, a framework that improves model robustness by employing contrastive learning to maximize similarity within positive pairs and minimize it between negative pairs, thereby enhancing the model’s ability to detect subtle differences in code changes. Additionally, feature fusion is used to combine semantic and expert features, enabling the model to capture richer contextual information. This integrated approach aims to improve the identification and resolution of code defects.
Results: JIT-CF was evaluated using the JIT-Defects4J dataset, which includes 23,379 code commits from 21 projects. The results indicate substantial performance improvements over seven state-of-the-art baselines, with enhancements of up to 13.9% in F1-score, 8% in AUC, and 11% in Recall@20%E. The study also explores the impact of specific customization enhancements, demonstrating the potential for improved just-in-time defect localization.
Conclusion: The proposed JIT-CF framework significantly advances the field of just-in-time defect prediction by effectively addressing the challenges encountered by pre-trained models in distinguishing code defects. The integration of contrastive learning and feature fusion not only enhances the model’s robustness but also leads to notable improvements in prediction accuracy, offering valuable insights for future applications in software development.

In this paper, a new method for extracting features from images is proposed based on fusion technology to recognize and classify image contents. To obtain the best features, the most important techniques used for extraction were combined, which include convolutional neural network (CNN), histogram of oriented gradient (HOG), and local binary pattern (LBP). This technique enables the model to fit features from the following aspects: integrating the features of shape, texture, scale, rotation, and translation. Homogeneous descriptors were also employed to feed the classification process, relying on a support vector machine (SVM) classifier. We applied this method to the oxford_flowers102 data set, due to the importance of flowers in the food chain of living organisms. Through experiments, we obtained results showing the superiority of the proposed method in terms of performance and accuracy over competing methods.

India's agricultural sector faces persistent challenges due to the prevalence of plant diseases, which severely impact crop quality and productivity, exacerbating the ongoing food supply crisis. Traditional methods of diagnosing plant diseases are often time-consuming, labor-intensive, and prone to inaccuracies, making it difficult for farmers to implement timely interventions. To address these issues, a forward-looking strategy utilizing artificial intelligence (AI) and machine learning (ML) has been proposed, aiming to revolutionize disease detection and management in agriculture. This involves the development of a comprehensive novel dataset named Leafsnap, which is uniquely sourced directly from real-world agricultural environments. This dataset ensures the authenticity and relevance of the data, reflecting the actual conditions faced by farmers. Leafsnap serves as a foundation for training advanced algorithmic models designed to identify patterns and symptoms indicative of various leaf diseases. The proposed system leverages a combination of cutting-edge AI and ML techniques, including convolutional neural networks (CNN), random forest (RF), support vector machines (SVM), and extreme gradient boosting (XGBoost) and logistic regression (LR). By integrating these advanced computational techniques into agricultural practices, the system aims to provide farmers with an efficient, reliable, and scalable solution for disease management. The ultimate goal is to foster agricultural sustainability by minimizing crop losses due to disease, thereby bolstering food security and supporting the livelihoods of farmers across India.

Classifying tumors from MRI scans is a key medical imaging and diagnosis task. Conventional feature-based methods and traditional machine learning algorithms are used for tumor classification, which limits their performance and generalization. A hybrid framework is implemented for the classification of brain tumors using MRIs. The framework contains three basic components, i.e., Feature Extraction, Feature Fusion, and Classification. The feature extraction module uses a convolutional neural network (CNN) to automatically extract high-level features from MRI images. The high-level features are combined with clinical and demographic features through a feature fusion module for better discriminative power. The Support vector machine (SVM) was employed to classify the fused features as class label tumors by a classification module. The proposed model obtained 90.67% accuracy, 94.67% precision, 83.82% recall and 83.71% f1-score. Experimental results demonstrate the superiority of our framework over those existing solutions and obtain exceptional accuracy rates compared to all other frequently operated models. This hybrid deep learning framework has promising performance for efficient and reproducible tumor classification within brain MRI scans.

In today's world, people are more prone to diseases due to food adulteration and pollution in the environment, and people have found a way of using herbal medicine as an alternative to allopathic medicine, especially since coronavirus disease 2019 (COVID-19). Medicinal plants are the source of herbal medicines that increase the immunity of humans. Medicinal plants are used in many applications, like pharmaceuticals, cosmetics, and drugs. Medicinal plants are of great importance, and hence this work presents a review of the medicinal plants grown in Karnataka State, India. The work also highlights species identification and disease detection of medicinal plants employing machine learning and deep learning approaches. The paper provides information about datasets available for various medicinal plant leaf images. The deep learning models used for species identification and disease detection in medicinal plants have been discussed along with the results.

Video object detection, a basic task in the computer vision field, is rapidly evolving and widely used. In recent years, deep learning methods have rapidly become widespread in the field of video object detection, achieving excellent results compared with those of traditional methods. However, the presence of duplicate information and abundant spatiotemporal information in video data poses a serious challenge to video object detection. Therefore, in recent years, many scholars have investigated deep learning detection algorithms in the context of video data and have achieved remarkable results. Considering the wide range of applications, a comprehensive review of the research related to video object detection is both a necessary and challenging task. This survey attempts to link and systematize the latest cutting-edge research on video object detection with the goal of classifying and analyzing video detection algorithms based on specific representative models. The differences and connections between video object detection and similar tasks are systematically demonstrated, and the evaluation metrics and video detection performance of nearly 40 models on two data sets are presented. Finally, the various applications and challenges facing video object detection are discussed.

Plant leaf diseases pose significant threats to global agriculture, leading to reduced crop yields and economic losses. Rapid and accurate disease detection is essential for timely interventions and sustainable farming practices. This study presents an innovative approach for plant leaf disease detection by integrating wavelet analysis, color, and texture features, coupled with autoencoder denoising and support vector machine (SVM) classification. Wavelet analysis is employed to extract multi-resolution features, capturing intricate details at different scales. Furthermore, color and texture characteristics are extracted to encompass a broad spectrum of visual information crucial for distinguishing diseases. The Autoencoder model helps to enhance the feature representation that mitigates the impact of noise and irrelevant data. The SVM classifier is utilized to learn complex patterns and accurately classify different disease classes. The combined model of wavelet, color, and texture attributes, in combination with autoencoder denoising and SVM classification, markedly enhances the precision and efficiency of disease detection in contrast to conventional methods. The system's performance is evaluated using a PlantVillage dataset, showcasing its adaptability to different plant species and disease types. The overall performance is obtained as 98.60%, 97.25%, 96.89%, and 97.20% in terms of accuracy, precision, recall, and F-Score, respectively.

Fire and smoke detection in today’s world is a must, especially in clustered areas where a quick response can prevent significant damages and save lives. Early detection plays a significant role in preventing the fire from spreading by alerting the emergency response personnel. It may not be possible to install traditional fire and smoke detectors everywhere. As a result, incorporating fire and smoke detection into existing closed circuit television (CCTV) systems in various places can provide a warning to the appropriate authorities, allowing for quick action to prevent the fire from spreading. This work aims in developing an early fire and smoke prediction model with CCTV footage images and video frames. The images and videos are collected from multiple datasets available online. A convolutional neural network (CNN) model is developed for early detection and prevention of the spreading of fire and compares it with transfer learning models ResNet50 and VGG19. The model obtain an ac...

Automation of objects labeling in aerial imagery is a computer vision task with numerous practical applications. Fields like energy exploration require an automated method to process a continuous stream of imagery on a daily basis. In this paper we propose a pipeline to tackle this problem using a stack of convolutional neural networks (U-Net architecture) arranged end-to-end. Each network works as post-processor to the previous one. Our model outperforms current state-of-the-art on two different datasets: Inria Aerial Image Labeling dataset and Massachusetts Buildings dataset each with different characteristics such as spatial resolution, object shapes and scales. Moreover, we experimentally validate computation time savings by processing sub-sampled images and later upsampling pixelwise labeling. These savings come at a negligible degradation in segmentation quality. Though the conducted experiments in this paper cover only aerial imagery, the technique presented is general and can handle other types of images.

Migratory waterfowl (i.e., ducks, geese, and swans) management relies on landscape bioenergetic models to inform on-the-ground habitat conditions and conservation practices. Therefore, conservation planners rely on accurate predictions of wetland habitats for waterfowl at regional scales. Unharvested flooded corn is a popular management tool on public and private lands that greatly increases landscape-level energy compared to other wetlands; thus, landscape bioenergetic models are particularly sensitive to these habitat features. Despite their importance to conservation planning and implementation, the abundance and distribution of unharvested flooded corn fields across North America is unknown. Furthermore, training data is difficult to collect and accurate predictions are challenging given their unique attributes and discreteness at landscape-level lens. Advances in multispectral imagery and deep learning algorithms may enable continuous and autonomous detection of these habitat features. Therefore, we conducted modeling experiments using training data of unharvested flooded corn fields in West Tennessee and multispectral imagery collected from Sentinel-2 satellite missions. We performed several experiments using individual band combination composites and/or vegetation indices to identify optimal bands using MRUNET architectures. We subsequently used 3 ensemble models of important individual networks. We found the use of multispectral bands was necessary and although the CIR composite and OSAVI index improved precision, the 12-band composite increased recall, the metric we were most interested in. Moreover, all ensembles exhibited poor performance. Here, we present results of our initial modeling experiments and suggest future modeling exercises including temporal image and vegetation index stacking using multi-modal and/or recurrent neural network architectures.

A persistent brain's neurological state is epilepsy, characterised by recurring seizure. Brain electrical activity is measured using EEG signals, which can be used to detect and diagnose significant brain problems such as Epilepsy, Autism, Alzheimer's etc. However, manual EEG data processing is time-consuming, requires highly skilled clinicians, and is associated with low inter-rater reliability (IRA). A computer-aided diagnosis approach for epileptic seizure detection from multichannel EEG recordings by fusing the time-frequency features and the deep learning features extracted from Convolutional Neural Network-Gated Recurrent Unit (CNN-GRU) model using canonical correlation analysis (CCA) method is provided in this study. Deep Learning features are extracted using CNN-GRU layers, motivated by recent advancements in image classification and optimised for use with EEG data. We have also extracted time-frequency features such as spectral entropies and Sub Band energies from Empirical mode decomposition (EMD) and Hilbert Marginal Spectrum (HMS). We used CHBMIT dataset to carry out the results and showed that the method proposed for fusing the time-frequency features and deep learning has given better performance.

Abstract. Nowadays an increasing number of people own mobile phones with built-in camera, able to take pictures. Thus, having a fast and fully automatic algorithm of image retrieval is considered a promising way to identify plant leaves on a mobile device. Our solution proposes a Support Vector Machine that provides a multi-class probability estimation with radial basis function kernel based on two descriptors: PHOG and a variant of HAAR. With our method we placed seventh among all the fully automatic methods who ...

The uncontrolled outburst in population has led to crowd gatherings in various public places causing panic and disaster in certain unpleasant and extreme conditions. A study on the analysis of crowd accumulation has been carried out for various reasons that include management of crowd, design of a well-planned public space, the possibility for surveillance at every area and transportation systems. A lot of disasters also occurs due to uncontrollable crowd behaviour and poor crowd management. It could result in loss of property, fatalities or casualties. To avoid this, the conduct of a crowd of people has been studied and analyzed to control their movement and traffic. Hence, in this research work, integrated multi-level feature fusion (IMFF) framework is designed to predict the behaviour; further classification based on the local region is carried out to enhance the prediction. In the case of multi-level feature fusion; first level feature fusion utilizes the motion and appearance; second-level feature fusion utilizes the spatial connection and third-level utilizes the temporal connections. Further, the classification approach is integrated based on the local region is used to enhance the crowd behaviour prediction in terms of accuracy and faster. Moreover, performance evaluation is carried out considering the two distinctive datasets.

Efficient road and building footprint extraction from satellite images are predominant in many remote sensing applications. However, precise segmentation map extraction is quite challenging due to the diverse building structures camouflaged by trees, similar spectral responses between the roads and buildings, and occlusions by heterogeneous traffic over the roads. Existing convolutional neural network (CNN)-based methods focus on either enriched spatial semantics learning for the building extraction or the fine-grained road topology extraction. The profound semantic information loss due to the traditional pooling mechanisms in CNN generates fragmented and disconnected road maps and poorly segmented boundaries for the densely spaced small buildings in complex surroundings. In this paper, we propose a novel attention-aware segmentation framework, Multi-Scale Supervised Dilated Multiple-Path Attention Network (MSSDMPA-Net), equipped with two new modules Dynamic Attention Map Guided Index Pooling (DAMIP) and Dynamic Attention Map Guided Spatial and Channel Attention (DAMSCA) to precisely extract the building footprints and road maps from remotely sensed images. DAMIP mines the salient features by employing a novel index pooling mechanism to retain important geometric information. On the other hand, DAMSCA simultaneously extracts the multi-scale spatial and spectral features. Besides, using dilated convolution and multi-scale deep supervision in optimizing MSSDMPA-Net helps achieve stellar performance. Experimental results over the seven benchmark building and road extraction datasets namely, Porto, Shanghai, Massachusetts Road, Massachusetts Building, Synthinel-1, WHU Satellite I and WHU Ariel Imagery dataset, ensures MSSDMPA-Net as the state-of-the-art (SOTA) method for building and road extraction as our method beats the next best alternatives by 5.

One of the major reasons for misclassification of multiplex actions during action recognition is the unavailability of complementary features that provide the semantic information about the actions. In different domains these features are present with different scales and intensities. In existing literature, features are extracted independently in different domains but the benefits from fusing these multidomain features are not realized. To address this challenge and to extract complete set of complementary information, in this paper, we propose a novel multidomain multimodal fusion framework that extracts complementary and distinct features from different domains of the input modality. We transform input inertial data into signal images, and then made the input modality multidomain and multimodal by transforming spatial domain information into frequency and time-spectrum domain using Discrete Fourier Transform (DFT) and Gabor wavelet transform (GWT) respectively. Features in different domains are extracted by Convolutional Neural networks (CNNs) and then fused by canonical correlation based fusion (CCF) for improving the accuracy of human action recognition. Experimental results on three inertial datasets show the superiority of the proposed method when compared to the state-of-the-art.

This paper attempts at improving the accuracy of Human Action Recognition (HAR) by fusion of depth and inertial sensor data. Firstly, we transform the depth data into Sequential Front view Images(SFI) and fine-tune the pre-trained AlexNet on these images. Then, inertial data is converted into Signal Images (SI) and another convolutional neural network (CNN) is trained on these images. Finally, learned features are extracted from both CNN, fused together to make a shared feature layer, and these features are fed to the classifier. We experiment with two classifiers, namely Support Vector Machines (SVM) and softmax classifier and compare their performances. The recognition accuracies of each modality, depth data alone and sensor data alone are also calculated and compared with fusion based accuracies to highlight the fact that fusion of modalities yields better results than individual modalities. Experimental results on UTD-MHAD and Kinect 2D datasets show that proposed method achieves state of the art results when compared to other recently proposed visual-inertial action recognition methods. Index Terms-Convolutional neural network, data augmentation, multimodal fusion.

A key step in seismic data processing is first break (FB) picking, or rather, determining the onset of the first seismic arrivals in seismic records. FB picking is tedious and time-consumingtask and robustness and efficient automatic method are essential. Many automated FB-picking algorithms already exist that reduce the dependence on human interaction. The goal of this project is to improve automated FB picking by capturing the expertise of FB picking through supervised training of deep neural network from the field of machine vision. We have evaluated several neural network architectures and found a seven layer U-net with skip connections to provide the best results on seismic shot gathers portioned into 64x64 windows. We applied the trained networks to unseen data with encouraging results, provided pre-processing is applied to make the unseen data similar to the training data. Additionally, by adding a small number of picked gathers from a new dataset to an existing training data...

To create an unbiased system, analysis of cricket videos is necessary. Detecting cricket performance based on the various cricket shots can also be useful for coaches and sports analysts. With the advances in hardware technologies and deep learning techniques, it has become feasible to apply these techniques in cricket shot detection. In this survey, we will be referring the research and implementation works of various scholars and find the best algorithm and technique suited for our purpose of classifying various images of batsmen playing different types of shots such as Cut Shot, Leg Glance, Straight Drive, Cover Drive, Sweep Shot. This survey will be very helpful for cricketers and coaches since it gives an insight into their type of shot played to a respective type of delivery. We have figured out that the CNN and transfer learning techniques are the most suited algorithm for our image classification purpose. This will be viewed in our future work.

This article proposes robust features fusion methodology for supervised palmprint recognition. The process of features fusion has been formulated according to the hybridization between robust morphological features of principal lines and the following features extractors: principal component analysis (PCA), transformed domain extractors, and invariant moments. The region of interest (ROI) has been extracted based on the identification of the valley points locations. Moreover, both morphological operations and edge detection have been applied in order to formulate the correct ROI. First, both length and slope of each extracted principal line have been estimated morphologically. Second, different features extraction techniques have been applied for the resultant ROI including (PCA), transformed domain, and invariant moments. Finally, seven supervised machine learning methods have been manipulated in order to achieve the highest palmprint recognition accuracy. The accuracy of the recognition was evaluated by measuring sensitivity, dice, precision, Jaccard coefficients, correlation, accuracy, and recognition time. Experimental findings show that, among all tested machine learning methods, the feed forward neural network with back propagation based on features vector of the fusion between invariant moments and principal lines morphology has achieved approximately 99.9% of successful palmprints recognition. K E Y W O R D S discrete cosine transform, discrete wavelet transform, invariant moments, principal component analysis, Hough transform, morphological features, transformed domain.

For the task of image classification, researchers work arduously to develop the next state-of-the-art (SOTA) model, each bench-marking their own performance against that of their predecessors and of their peers. Unfortunately, the metric used most frequently to describe a model's performance, average categorization accuracy, is often used in isolation. As the number of classes increases, such as in fine-grained visual categorization (FGVC), the amount of information conveyed by average accuracy alone dwindles. While its most glaring weakness is its failure to describe the model's performance on a class-by-class basis, average accuracy also fails to describe how performance may vary from one trained model of the same architecture, on the same dataset, to another (both averaged across all categories and at the per-class level). We first demonstrate the magnitude of these variations across models and across class distributions based on attributes of the data, comparing results on different visual domains and different per-class image distributions, including long-tailed distributions and few-shot subsets. We then analyze the impact various FGVC methods have on overall and per-class variance. From this analysis, we both highlight the importance of reporting and comparing methods based on information beyond overall accuracy, as well as point out techniques that mitigate variance in FGVC results.

Dramatic appearance variation due to pose constitutes a great challenge in fine-grained recognition, one which recent methods using attention mechanisms or second-order statistics fail to adequately address. Modern CNNs typically lack an explicit understanding of object pose and are instead confused by entangled pose and appearance. In this paper, we propose a unified object representation built from a hierarchy of pose-aligned regions. Rather than representing an object by regions aligned to image axes, the proposed representation characterizes appearance relative to the object's pose using pose-aligned patches whose features are robust to variations in pose, scale and rotation. We propose an algorithm that performs pose estimation and forms the unified object representation as the concatenation of hierarchical pose-aligned regions features, which is then fed into a classification network. The proposed algorithm surpasses the performance of other approaches, increasing the state-of-the-art by nearly 2% on the widely-used CUB-200 [42] dataset and by more than 8% on the much larger NABirds [41] dataset. The effectiveness of this paradigm relative to competing methods suggests the critical importance of disentangling pose and appearance for continued progress in fine-grained recognition.

To create an unbiased system, analysis of cricket videos is necessary. Detecting cricket performance based on the various cricket shots can also be useful for coaches and sports analysts. With the advances in hardware technologies and deep learning techniques, it has become feasible to apply these techniques in cricket shot detection. In this survey, we will be referring the research and implementation works of various scholars and find the best algorithm and technique suited for our purpose of classifying various images of batsmen playing different types of shots such as Cut Shot, Leg Glance, Straight Drive, Cover Drive, Sweep Shot. This survey will be very helpful for cricketers and coaches since it gives an insight into their type of shot played to a respective type of delivery. We have figured out that the CNN and transfer learning techniques are the most suited algorithm for our image classification purpose. This will be viewed in our future work.

The human hand has been considered a promising component for biometric-based identification and authentication systems for many decades. In this paper, hand side recognition framework is proposed based on deep learning and biometric authentication using the hashing method. The proposed approach performs in three phases: (a) hand image segmentation and enhancement by morphological filtering, automatic thresholding, and active contour deformation, (b) hand side recognition based on deep Convolutional Neural Networks (CNN), and (c) biometric authentication based on the hashing method. The proposed framework is evaluated using a very large hand dataset, which consists of 11076 hand images, including left/ right and dorsal/ palm hand images for 190 persons. Finally, the experimental results show the efficiency of the proposed framework in both dorsal-palm and left-right recognition with an average accuracy of 96.24 and 98.26, respectively, using a completely automated computer program.

Vehicle re-identification (Re-Id) is the key module in an intelligent transportation system (ITS). Due to its versatile applicability in metropolitan cities, this task has received increasing attention these days. It aims to identify whether the specific vehicle has already appeared over the surveillance network or not. Mostly, the vehicle Re-Id method are evaluated on a single dataset, in which training and testing of the model is performed on the same dataset. However in practice, this negatively effects model generalization ability due to biased datasets along with the significant difference between training and testing data; hence, the model becomes weak in a practical environment. To demonstrate this issue, we have empirically shown that the current vehicle Re-Id datasets are usually strongly biased. In this regard, we also conduct an extensive study on the cross and the same dataset to examine the impact on the performance of the vehicle Re-Id system, considering existing meth...

Automatic human action recognition is a challenging and largely explored domain. In this work, we focus on action segmentation with Hough Transform paradigm and more precisely with Deeply Optimised Hough Transform (DOHT). First, we apply DOHT on video sequences using the well-known dense trajectories features and then, we propose to extend the method to efficiently merge information coming from various sensors. We have introduced three different ways to perform fusion, depending on the level at which information is merged. Advantages and disadvantages of these solutions are presented from the performance point of view and also according to the ease of use. Thus, one of the fusion level has the advantage to stay suitabe even if one or more sensors is out of order or disturbed.

In the past two decades, human action recognition has been among the most challenging tasks in the field of computer vision. Recently, extracting accurate and cost-efficient skeleton information became available thanks to the cutting edge deep learning algorithms and low-cost depth sensors. In this paper, we propose a novel framework to recognize human actions using 3D skeleton information. The main components of the framework are pose representation and encoding. Assuming that human actions can be represented by spatiotemporal poses, we define a pose descriptor consisting of three elements. The first element contains the normalized coordinates of the raw skeleton joints information. The second element contains the temporal displacement information relative to a predefined temporal offset and the third element keeps the displacement information pertinent to the previous timestamp in the temporal resolution. The final descriptor of the whole sequence is the concatenation of frame-wise descriptors. To avoid the problems regarding high dimensionality, Principal Component Analysis (PCA) is applied on the descriptors. The resulted descriptors are encoded with Fisher Vector (FV) representation before they get trained with an Extreme Learning Machine (ELM). The performance of the proposed framework is evaluated by three public benchmark datasets. The proposed method achieved competitive results compared to the other methods in the literature.

The ability to correctly classify and retrieve apparel images has a variety of applications important to e-commerce, online advertising, internet search, and visual surveillance industry. In this work, we propose a robust framework for fine-grained apparel classification, in-shop and cross-domain retrieval which eliminates the requirement of rich annotations like bounding boxes and human-joints or clothing landmarks, and training of bounding box/ key-landmark detector for the same. Factors such as subtle appearance differences, variations in human poses, different shooting angles, apparel deformations, and self-occlusion add to the challenges in classification and retrieval of apparel items. Cross-domain retrieval is even harder due to the presence of large variation between online shopping images, usually taken in ideal lighting, pose, positive angle and clean background as compared with street photos captured by users in complicated conditions with poor lighting, cluttered scenes, and complex background. Our framework utilizes compact bilinear CNN [11] with tensor sketch algorithm to generate embeddings that capture local pairwise feature interactions in a translationally invariant manner. For apparel classification, we pass the obtained feature embeddings through a softmax classifier, while, the in-shop and cross-domain retrieval pipelines use a tripletloss based optimization approach and deploy three compact BCNNs, with a ranking loss such that squared Euclidean distance between embeddings measures the dissimilarity between the images. Unlike previous settings that relied on bounding box, key clothing landmarks or human joint detectors to assist the final deep classifier, proposed framework can be trained directly on the provided category labels or generated triplets for triplet loss optimization. Lastly,

In the present study, a multimodal biometric authentication method is presented to confirm the identity of a person based on his face and iris features. This method depends on multiple biometric techniques that combine face and iris (left and right) features to recognize. The authors have designed and applied a system to identify people. It depends on extracting the features of the face using Rectangle Histogram of Oriented Gradient (R-HOG). The study applies a feature-level fusion using a novel fusion method which employs both the canonical correlation process and the proposed serial concatenation. A deep belief network was used for the recognition process. The performance of the proposed systems was validated and evaluated through a set of experiments on SDUMLA-HMT database. The results were compared with others, and have shown that the fusion time has been reduced by about 34.5%. The proposed system has also succeeded in achieving a lower equal error rate (EER), and a recognition...

This work addresses the problem of vehicle identification through non-overlapping cameras. As our main contribution, we introduce a novel dataset for vehicle identification, called Vehicle-Rear, that contains more than three hours of high-resolution videos, with accurate information about the make, model, color and year of nearly 3,000 vehicles, in addition to the position and identification of their license plates. To explore our dataset we design a two-stream Convolutional Neural Network (CNN) that simultaneously uses two of the most distinctive and persistent features available: the vehicle's appearance and its license plate. This is an attempt to tackle a major problem: false alarms caused by vehicles with similar designs or by very close license plate identifiers. In the first network stream, shape similarities are identified by a Siamese CNN that uses a pair of low-resolution vehicle patches recorded by two different cameras. In the second stream, we use a CNN for Optical Character Recognition (OCR) to extract textual information, confidence scores, and string similarities from a pair of high-resolution license plate patches. Then, features from both streams are merged by a sequence of fully connected layers for decision. In our experiments, we compared the two-stream network against several well-known CNN architectures using single or multiple vehicle features. The architectures, trained models, and dataset are publicly available at https://github.com/icarofua/vehicle-rear. INDEX TERMS Vehicle identification, vehicle matching, multi-stream neural networks, feature fusion.

The signature recognition is a difficult process as it requires several phases. A failure in a phase will significantly reduce the recognition accuracy. Artificial Neural Network (ANN) believed to be used to assist in the recognition or classification of the signature. In this study, the ANN algorithm used is Back Propagation. A mechanism to adaptively adjust the learning rate is developed to improve the system accuracy. The purpose of this study is to conduct the recognition of a number of signatures so that can be known whether the recognition which is done by using the Back Propagation is appropriate or not. The testing results performed by using learning rate of 0.64, the number of iterations is 100, and produces an accuracy value of 63%.

The ability to correctly classify and retrieve apparel images has a variety of applications important to e-commerce, online advertising, internet search, and visual surveillance industry. In this work, we propose a robust framework for fine-grained apparel classification, in-shop and cross-domain retrieval which eliminates the requirement of rich annotations like bounding boxes and human-joints or clothing landmarks, and training of bounding box/ key-landmark detector for the same. Factors such as subtle appearance differences, variations in human poses, different shooting angles, apparel deformations, and self-occlusion add to the challenges in classification and retrieval of apparel items. Cross-domain retrieval is even harder due to the presence of large variation between online shopping images, usually taken in ideal lighting, pose, positive angle and clean background as compared with street photos captured by users in complicated conditions with poor lighting, cluttered scenes, and complex background. Our framework utilizes compact bilinear CNN [11] with tensor sketch algorithm to generate embeddings that capture local pairwise feature interactions in a translationally invariant manner. For apparel classification, we pass the obtained feature embeddings through a softmax classifier, while, the in-shop and cross-domain retrieval pipelines use a tripletloss based optimization approach and deploy three compact BCNNs, with a ranking loss such that squared Euclidean distance between embeddings measures the dissimilarity between the images. Unlike previous settings that relied on bounding box, key clothing landmarks or human joint detectors to assist the final deep classifier, proposed framework can be trained directly on the provided category labels or generated triplets for triplet loss optimization. Lastly,

In recent years, systems of ear recognition are considered a significant topic of research in the biometrics field. In such systems, the models of machine learning represent a principal part in order to recognise humans’ identities by using their ear images. In this paper, a system of ear recognition is proposed by using random forest (RF) and histograms of oriented gradients (HOG) techniques. The HOG is used to extract features from ear images. Subsequently, these extracted features will be fed to the RF classifier to classify the ear images with respect to the classes. In this study, the ear images have been selected from the Indian Institute of Technology Delhi, second version (IITD II). The performance of the proposed system has evaluated by using different evaluation measures such as accuracy, specificity, and G-mean. The experimental results show that the proposed system for ear recognition obtains accuracy up to 99.69%. Furthermore, this system archives 99.84% and 80.78% for ...

This work addresses the problem of vehicle identification through non-overlapping cameras. As our main contribution, we introduce a novel dataset for vehicle identification, called Vehicle-Rear, that contains more than three hours of high-resolution videos, with accurate information about the make, model, color and year of nearly 3,000 vehicles, in addition to the position and identification of their license plates. To explore our dataset we design a two-stream Convolutional Neural Network (CNN) that simultaneously uses two of the most distinctive and persistent features available: the vehicle's appearance and its license plate. This is an attempt to tackle a major problem: false alarms caused by vehicles with similar designs or by very close license plate identifiers. In the first network stream, shape similarities are identified by a Siamese CNN that uses a pair of low-resolution vehicle patches recorded by two different cameras. In the second stream, we use a CNN for Optical Character Recognition (OCR) to extract textual information, confidence scores, and string similarities from a pair of high-resolution license plate patches. Then, features from both streams are merged by a sequence of fully connected layers for decision. In our experiments, we compared the two-stream network against several well-known CNN architectures using single or multiple vehicle features. The architectures, trained models, and dataset are publicly available at https://github.com/icarofua/vehicle-rear. INDEX TERMS Vehicle identification, vehicle matching, multi-stream neural networks, feature fusion.

Existing approaches for image-based Automatic Meter Reading (AMR) have been evaluated on images captured in well-controlled scenarios. However, real-world meter reading presents unconstrained scenarios that are way more challenging due to dirt, various lighting conditions, scale variations, in-plane and out-of-plane rotations, among other factors. In this work, we present an end-to-end approach for AMR focusing on unconstrained scenarios. Our main contribution is the insertion of a new stage in the AMR pipeline, called corner detection and counter classification, which enables the counter region to be rectifiedas well as the rejection of illegible/faulty meters-prior to the recognition stage. We also introduce a publicly available dataset, called Copel-AMR, that contains 12,500 meter images acquired in the field by the service company's employees themselves, including 2,500 images of faulty meters or cases where the reading is illegible due to occlusions. Experimental evaluation demonstrates that the proposed system, which has three networks operating in a cascaded mode, outperforms all baselines in terms of recognition rate while still being quite efficient. Moreover, as very few reading errors are tolerated in real-world applications, we show that our AMR system achieves impressive recognition rates (i.e., ≥99%) when rejecting readings made with lower confidence values. INDEX TERMS Computer vision, image-based automatic meter reading, unconstrained scenarios.

Classification of electroencephalography (EEG) signals for brain-computer interface has great impact on people having various kinds of physical disabilities. Motor imagery EEG signals of hand and leg movement classification can help people whose limbs are replaced by prosthetics. In this paper, random subspace ensemble network with variable length feature sampling has been proposed for improving the prediction accuracy of motor imagery EEG signal classification. The method has been tested on eight different subjects and a hybrid dataset of two subjects data combined. Discrete wavelet transform based de-noising scheme has been adopted to remove artifacts from the EEG signal. For sub-band selection, dual-tree complex wavelet Transform has been employed. Mutual information scoring has been used for univariate feature selection from the feature space. A comparative analysis has been carried out where random subspace ensemble network outperformed other classification models. The maximum ...

Object appearances often change dramatically with pose variations. This creates a challenge for embedding schemes that seek to map instances with the same object ID to locations that are as close as possible. This issue becomes significantly heightened in complex computer vision tasks such as re-identification(re-id). In this paper, we suggest these dramatic appearance changes are indications that an object ID is composed of multiple natural groups and it is counter-productive to forcefully map instances from different groups to a common location. This leads us to introduce Relation Preserving Triplet Mining (RPTM), a feature matching guided triplet mining scheme, that ensures triplets will respect the natural sub-groupings within an object ID. We use this triplet mining mechanism to establish a poseaware, well-conditioned triplet cost function. This allows a single network to be trained with fixed parameters across three challenging benchmarks, while still providing stateof-the-art...

In recent years, systems of ear recognition are considered a significant topic of research in the biometrics field. In such systems, the models of machine learning represent a principal part in order to recognise humans’ identities by using their ear images. In this paper, a system of ear recognition is proposed by using random forest (RF) and histograms of oriented gradients (HOG) techniques. The HOG is used to extract features from ear images. Subsequently, these extracted features will be fed to the RF classifier to classify the ear images with respect to the classes. In this study, the ear images have been selected from the Indian Institute of Technology Delhi, second version (IITD II). The performance of the proposed system has evaluated by using different evaluation measures such as accuracy, specificity, and G-mean. The experimental results show that the proposed system for ear recognition obtains accuracy up to 99.69%. Furthermore, this system archives 99.84% and 80.78% for ...

In this paper, we propose a method for user Finger Vein Authentication (FVA) as a biometric system. Using the discriminative features for classifying theses finger veins is one of the main tips that make difference in related works, Thus we propose to learn a set of representative features, based on autoencoders. We model the user finger vein using a Gaussian distribution. Experimental results show that our algorithm perform like a state-of-the-art on SDUMLA-HMT benchmark.

This paper presents a new proposal of an efficient computational model of face recognition which uses cues from the distributed face recognition mechanism of the brain, and by gathering engineering equivalent of these cues from existing literature. Three distinct and widely used features – Histogram of Oriented Gradients (HOG), Local Binary Patterns (LBP), and Principal components (PCs) extracted from target images are used in a manner which is simple, and yet effective. The HOG and LBP features further undergo principal component analysis for dimensionality reduction. Our model uses multi-layer perceptrons (MLP) to classify these three features and fuse them at the decision level using sum rule. A computational theory is first developed by using concepts from the information processing mechanism of the brain. Extensive experiments are carried out using ten publicly available datasets to validate our proposed model’s performance in recognizing faces with extreme variation of illumin...

In this work, advanced learning and moving window-based methods have been used for epileptic seizure detection. Epilepsy is a disorder of the central nervous system and roughly affects 50 million people worldwide. The most common noninvasive tool for studying the brain activity of an epileptic patient is the electroencephalogram. Accurate detection of seizure onset is still an elusive work. Electroencephalogram signals belonging to pediatric patients from Children's Hospital Boston, Massachusetts Institute of Technology have been used in this work to validate the proposed method. For determining between seizure and non-seizure signals, feature extraction techniques based on time-domain, frequency domain, time-frequency domain have been used. Four different methods (decision tree, random forest, artificial neural network, and ensemble learning) have been studied and their performances have been compared using different statistical measures. The test sample technique has been used for the validation of all seizure detection methods. The results show better performance by random forest among all the four classifiers with an accuracy, sensitivity, and specificity of 91.9%, 94.1%, and 89.7% respectively. The proposed method is suggested as an improved method because it is not channel specific, not patient specific and has a promising accuracy in detecting epileptic seizure.