Fully Convolutional Neural Network (FCN)

More than two-thirds of the world's population rely on rice or wheat as staple foods, which are grown in various Asian countries. Diseases affecting rice leaves can disrupt growth, reduce yields, and cause famine in some areas. Therefore, a quick and accurate recognition method is necessary to minimize losses. This article focuses on eight types of rice leaf diseases using data consisting of approximately 110 images for each disease type, with enhanced image quality to achieve better results. The study applies a convolutional neural network (CNN) model integrated into an Android mobile application, achieving a training accuracy of 86.56% and a validation accuracy of 93.75%. Comparative experiments demonstrate that the model can be effectively implemented in mobile applications for accurately detecting rice leaf diseases, providing a reliable solution for field detection. This method not only helps farmers identify diseases more quickly but also has the potential to reduce crop losses caused by leaf diseases.

The retinal vascular condition is a reliable biomarker of several ophthalmologic and cardiovascular diseases, so automatic vessel segmentation may be crucial to diagnose and monitor them. In this paper, we propose a novel method that combines the multiscale analysis provided by the Stationary Wavelet Transform with a multiscale Fully Convolutional Neural Network to cope with the varying width and direction of the vessel structure in the retina. Our proposal uses rotation operations as the basis of a joint strategy for both data augmentation and prediction, which allows us to explore the information learned during training to refine the segmentation. The method was evaluated on three publicly available databases, achieving an average accuracy of 0.9576, 0.9694, and 0.9653, and average area under the ROC curve of 0.9821, 0.9905, and 0.9855 on the DRIVE, STARE, and CHASE DB1 databases, respectively. It also appears to be robust to the training set and to the inter-rater variability, which shows its potential for real-world applications.

The goal of video cosegmentation is to jointly extract the common foreground regions and/or objects from a set of videos. In this paper, we present an approach for video cosegmentation that uses graphbased hierarchical clustering as its basic component. Actually, in this work, video cosegmentation problem is transformed into a graph-based clustering problem in which a cluster represents a set of similar supervoxels belonging to the analyzed videos. Our graph-based Hierarchical Video Cosegmentation method (or HVC) is divided in two main parts: (i) supervoxel generation and (ii) supervoxel correlation. The former explores only intra-video similarities, while the latter seeks to determine relationships between supervoxels belonging to the same video or to distinct videos. Experimental results provide comparison between HVC and other methods from the literature on two well known datasets, showing that HVC is a competitive one. HVC outperforms on average all the compared methods for one dataset; and it was the second best for the other one. Actually, HVC is able to produce good quality results without being too computational expensive, taking less than 50% of the time spent by any other approach.

This paper proposes a segmentation method of infection regions in the lung from CT volumes of COVID-19 patients. COVID-19 spread worldwide, causing many infected patients and deaths. CT imagebased diagnosis of COVID-19 can provide quick and accurate diagnosis results. An automated segmentation method of infection regions in the lung provides a quantitative criterion for diagnosis. Previous methods employ whole 2D image or 3D volume-based processes. Infection regions have a considerable variation in their sizes. Such processes easily miss small infection regions. Patch-based process is effective for segmenting small targets. However, selecting the appropriate patch size is difficult in infection region segmentation. We utilize the scale uncertainty among various receptive field sizes of a segmentation FCN to obtain infection regions. The receptive field sizes can be defined as the patch size and the resolution of volumes where patches are clipped from. This paper proposes an infection segmentation network (ISNet) that performs patchbased segmentation and a scale uncertainty-aware prediction aggregation method that refines the segmentation result. We design ISNet to segment infection regions that have various intensity values. ISNet has multiple encoding paths to process patch volumes normalized by multiple intensity ranges. We collect prediction results generated by ISNets having various receptive field sizes. Scale uncertainty among the prediction results is extracted by the prediction aggregation method. We use an aggregation FCN to generate a refined segmentation result considering scale uncertainty among the predictions. In our experiments using 199 chest CT volumes of COVID-19 cases, the prediction aggregation method improved the dice similarity score from 47.6% to 62.1%.

This paper proposes an automated segmentation method of infection and normal regions in the lung from CT volumes of COVID-19 patients. From December 2019, novel coronavirus disease 2019 (COVID-19) spreads over the world and giving significant impacts to our economic activities and daily lives. To diagnose the large number of infected patients, diagnosis assistance by computers is needed. Chest CT is effective for diagnosis of viral pneumonia including COVID-19. A quantitative analysis method of condition of the lung from CT volumes by computers is required for diagnosis assistance of COVID-19. This paper proposes an automated segmentation method of infection and normal regions in the lung from CT volumes using a COVID-19 segmentation fully convolutional network (FCN). In diagnosis of lung diseases including COVID-19, analysis of conditions of normal and infection regions in the lung is important. Our method recognizes and segments lung normal and infection regions in CT volumes. To segment infection regions that have various shapes and sizes, we introduced dense pooling connections and dilated convolutions in our FCN. We applied the proposed method to CT volumes of COVID-19 cases. From mild to severe cases of COVID-19, the proposed method correctly segmented normal and infection regions in the lung. Dice scores of normal and infection regions were 0.911 and 0.753, respectively.

Recently many graph-based salient region/object detection methods have been developed. They are rather effective for still images. However, little attention has been paid to salient region detection in videos. This paper addresses salient region detection in videos. A unified approach towards graph construction for salient object detection in videos is proposed. The proposed method combines static appearance and motion cues to construct graph, enabling a direct extension of original graphbased salient region detection to video processing. To maintain coherence in both intra-and inter-frames, a spatial-temporal smoothing operation is proposed on a structured graph derived from consecutive frames. The effectiveness of the proposed method is tested and validated using seven videos from two video datasets.

This paper presents a video object segmentation method which jointly uses motion boundary and convolutional neural network (CNN)-based class-level maps to carry out the co-segmentation of the frames. The key characteristic of the proposed approach is a combination of those two sources of information to create initial object and background regions. These regions are employed within the co-segmentation energy function. The motion boundary map detects the areas which contain the object movement, and the CNN-based class saliency map determines the regions with more impact on acquiring the correct network classification. The proposed approach can be implemented on unconstrained natural videos which include changes in an object's appearance, rapidly moving background, object deformation in non-rigid moving, rapid camera motion and even the existence of a static object. Experimental results on two challenging datasets (i.e., Davis and SegTrackv2 datasets) demonstrate the competitive performance of the proposed method compared with the state-ofthe-art approaches.

We address the problem of multi-view video segmentation of dynamic scenes in general and outdoor environments with possibly moving cameras. Multi-view methods for dynamic scenes usually rely on geometric calibration to impose spatial shape constraints between viewpoints. In this paper, we show that the calibration constraint can be relaxed while still getting competitive segmentation results using multi-view constraints. We introduce new multi-view cotemporality constraints through motion correlation cues, in addition to common appearance features used by cosegmentation methods to identify co-instances of objects. We also take advantage of learning based segmentation strategies by casting the problem as the selection of monocular proposals that satisfy multi-view constraints. This yields a fully automated method that can segment subjects of interest without any particular pre-processing stage. Results on several challenging outdoor datasets demonstrate the feasibility and robustness of our approach.

In this paper we introduce a novel method for segmentation that can benefit from general semantics of Convolutional Neural Network (CNN). Our segmentation proposes visually and semantically coherent image segments. We use binary encoding of CNN features to overcome the difficulty of the clustering on the high-dimensional CNN feature space. These binary encoding can be embedded into the CNN as an extra layer at the end of the network. This results in real-time segmentation. To the best of our knowledge our method is the first attempt on general semantic image segmentation using CNN. All the previous papers were limited to few number of category of the images (e.g. PASCAL VOC). Experiments show that our segmentation algorithm outperform the state-of-the-art non-semantic segmentation methods by a large margin.

This paper addresses the problem of image segmentation by iterative region aggregations starting from an initial superpixel decomposition. Classical approaches for this task compute superpixel similarity using distance measures between superpixel descriptor vectors. This usually poses the well-known problem of the semantic gap and fails to properly aggregate visually non-homogeneous superpixels that belong to the same high-level object. This work proposes to use random forests to learn the merging probability between adjacent superpixels in order to overcome the aforementioned issues. Compared to existing works, this approach learns the fusion rules without explicit similarity measure computation. We also introduce a new superpixel context descriptor to strengthen the learned characteristics towards better similarity prediction. Image segmentation is then achieved by iteratively merging the most similar superpixel pairs selected using a similarity weighting objective function. Experimental results of our approach on four datasets including DAVIS 2017 and ISIC 2018 show its potential compared to state-of-the-art approaches.

Purpose:
The purpose of this paper is to present a fully automated abdominal artery segmentation method from a CT volume. Three-dimensional (3D) blood vessel structure information is important for diagnosis and treatment. Information about blood vessels (including arteries) can be used in patient-specific surgical planning and intra-operative navigation. Since blood vessels have large inter-patient variations in branching patterns and positions, a patient-specific blood vessel segmentation method is necessary. Even though deep-learning-based segmentation methods provide good segmentation accuracy among large organs, small organs such as blood vessels are not well segmented. We propose a deep-learning-based abdominal artery segmentation method from a CT volume. Because the artery is one of small organs that is difficult to segment, we introduced an original training sample generation method and a three-plane segmentation approach to improve segmentation accuracy.

Method:
Our proposed method segments abdominal arteries from an abdominal CT volume with a fully convolutional network (FCN). To segment small arteries, we employ a 2D patch-based segmentation method and an area imbalance reduced training patch generation (AIRTPG) method. AIRTPG adjusts patch number imbalances between patches with artery regions and patches without them. These methods improved the segmentation accuracies of small artery regions. Furthermore, we introduced a three-plane segmentation approach to obtain clear 3D segmentation results from 2D patch-based processes. In the three-plane approach, we performed three segmentation processes using patches generated on axial, coronal, and sagittal planes and combined the results to generate a 3D segmentation result.

Results:
The evaluation results of the proposed method using 20 cases of abdominal CT volumes show that the averaged F-measure, precision, and recall rates were 87.1%, 85.8%, and 88.4%, respectively. This result outperformed our previous automated FCN-based segmentation method. Our method offers competitive performance compared to the previous blood vessel segmentation methods from 3D volumes.

Conclusions:
We developed an abdominal artery segmentation method using FCN. The 2D patch-based and AIRTPG methods effectively segmented the artery regions. In addition, the three-plane approach generated good 3D segmentation results.

This paper presents a fully automated atlas-based pancreas segmentation method from CT volumes utilizing 3D fully convolutional network (FCN) feature-based pancreas localization. Segmentation of the pancreas is difficult because it has larger inter-patient spatial variations than other organs. Previous pancreas segmentation methods failed to deal with such variations. We propose a fully automated pancreas segmentation method that contains novel localization and segmentation. Since the pancreas neighbors many other organs, its position and size are strongly related to the positions of the surrounding organs. We estimate the position and the size of the pancreas (localized) from global features by regression forests. As global features, we use intensity differences and 3D FCN deep learned features, which include automatically extracted essential features for segmentation. We chose 3D FCN features from a trained 3D U-Net, which is trained to perform multi-organ segmentation. The global features include both the pancreas and surrounding organ information. After localization, a patient-specific probabilistic atlas-based pancreas segmentation is performed. In evaluation results with 146 CT volumes, we achieved 60.6% of the Jaccard index and 73.9% of the Dice overlap.

This paper proposes an automated segmentation method of infection and normal regions in the lung from CT volumes of COVID-19 patients. From December 2019, novel coronavirus disease 2019 (COVID-19) spreads over the world and giving significant impacts to our economic activities and daily lives. To diagnose the large number of infected patients, diagnosis assistance by computers is needed. Chest CT is effective for diagnosis of viral pneumonia including COVID-19. A quantitative analysis method of condition of the lung from CT volumes by computers is required for diagnosis assistance of COVID-19. This paper proposes an automated segmentation method of infection and normal regions in the lung from CT volumes using a COVID-19 segmentation fully convolutional network (FCN). In diagnosis of lung diseases including COVID-19, analysis of conditions of normal and infection regions in the lung is important. Our method recognizes and segments lung normal and infection regions in CT volumes. To segment infection regions that have various shapes and sizes, we introduced dense pooling connections and dilated convolutions in our FCN. We applied the proposed method to CT volumes of COVID-19 cases. From mild to severe cases of COVID-19, the proposed method correctly segmented normal and infection regions in the lung. Dice scores of normal and infection regions were 0.911 and 0.753, respectively.

When the foreground objects have variegated appearance and/or manifest articulated motion, not to mention the momentary occlusions by other unintended objects, a segmentation method based on single video and a bottom-up approach is often insufficient for their extraction. In this paper, we present a video co-segmentation method to address the aforementioned challenges. Departing from the objectness attributes and motion coherence used by traditional figure-ground separation methods, we place central importance in the role of "common fate", that is, the different parts of the foreground should persist together in all the videos. To accomplish this idea, we first extract seed superpixels by a motion-based figure/ground segmentation method. We then formulate a set of linkage constraints between these superpixels based on whether they exhibit the characteristics of common fate or not. An iterative constrained clustering algorithm is then proposed to trim away the incorrect and accidental linkage relationships. The clustering algorithm also performs automatic model selection to estimate the number of individual objects in the foreground (e.g. male and female birds in courtship), while at the same time binding the parts of a variegated object together in a unified whole. Finally, a multiclass labeling Markov randome field is used to obtain a refined segmentation result. Our experimental results on two datasets show that our method successfully addresses the challenges in the extraction of complex foreground and outperforms the state-of-the-art video segmentation and co-segmentation methods.

Image co-segmentation is jointly segmenting two or more images sharing common foreground objects. In this paper, we propose a novel graph convolution neural network (graph CNN) based end-to-end model for performing co-segmentation. At the beginning, each input image is over-segmented into a set of superpixels. Next, a weighted graph is formed using the over-segmented images exploiting spatial adjacency and both intra-image and inter-image feature similarities among the image superpixels (nodes). Subsequently, the proposed network, consisting of graph convolution layers followed by node classification layers, classifies each superpixel either into the common foreground or its complement. During training, along with the cosegmentation network, an additional network is introduced to exploit the corresponding semantic labels, and the two networks share the same weights in graph convolution layers. The whole model is learned in an end-to-end fashion using a novel cost function comprised of a superpixel wise binary cross entropy and a multi-label cross entropy. We also use empirical class probabilities in the loss function to deal with class imbalance. Experimental results reflect that the proposed technique is very competitive with the state-of-the-art methods on two challenging datasets, Internet and Pascal-VOC.

Some cognitive research has discovered that humans accomplish event segmentation as a side effect of event anticipation. Inspired by this discovery, we propose a simple yet effective end-to-end self-supervised learning framework for event segmentation/boundary detection. Unlike the mainstream clustering-based methods, our framework exploits a transformer-based feature reconstruction scheme to detect event boundary by reconstruction errors. This is consistent with the fact that humans spot new events by leveraging the deviation between their prediction and what is actually perceived. Thanks to their heterogeneity in semantics, the frames at boundaries are difficult to be reconstructed (generally with large reconstruction errors), which is favorable for event boundary detection. Additionally, since the reconstruction occurs on the semantic feature level instead of pixel level, we develop a temporal contrastive feature embedding module to learn the semantic visual representation for fr...

We present a novel solution to the problem of detecting common actions in time series of motion capture data and videos. Given two action sequences, our method discovers all pairs of common subsequences, i.e. subsequences that represent the same or similar action. This is achieved in a completely unsupervised manner, i.e., without any prior knowledge of the type of actions, their number and their duration. These common subsequences (commonalities) may be located anywhere in the original sequences, may differ in duration and may be performed under different conditions e.g., by a different actor. The proposed method performs a very efficient graph-based search on the matrix of pairwise distances of frames of the two sequences. This search is supported by an objective function that captures the trade off between the similarity of the common subsequences and their lengths. The proposed method has been evaluated quantitatively on challenging datasets and in comparison to state of the art approaches. The obtained results demonstrate that the proposed method outperforms the state of the art methods both in the quality of the obtained solutions and in computational performance.

This work proposes a novel attentive graph neural network (AGNN) for zero-shot video object segmentation (ZVOS). The suggested AGNN recasts this task as a process of iterative information fusion over video graphs. Specifically, AGNN builds a fully connected graph to efficiently represent frames as nodes, and relations between arbitrary frame pairs as edges. The underlying pair-wise relations are described by a differentiable attention mechanism. Through parametric message passing, AGNN is able to efficiently capture and mine much richer and higher-order relations between video frames, thus enabling a more complete understanding of video content and more accurate foreground estimation. Experimental results on three video segmentation datasets show that AGNN sets a new state-ofthe-art in each case. To further demonstrate the generalizability of our framework, we extend AGNN to an additional task: image object co-segmentation (IOCS). We perform experiments on two famous IOCS datasets and observe again the superiority of our AGNN model. The extensive experiments verify that AGNN is able to learn the underlying semantic/appearance relationships among video frames or related images, and discover the common objects.

Spider mites are important pests that cause severe economic damage to cotton. They feed on underside of leaves, piercing the chloroplast-containing cells, resulting in foliar damage and yield reduction. This paper proposed a two-stage classification approach for mite-infestation detection based on machine learning methods. Two cotton fields were selected for study, and the UAV imagery collection and concurrent ground investigation were conducted on July 20-21th, 2017. Mosaicking and geo-registration were performed on the collected multispectral imagery. Support Vector Machine (SVM) was used for scene classification, and a transferred Convolutional Neural Network (CNN) was applied for mite-infestation identification. Experimental results showed that our method outperformed others in terms of accuracy, which demonstrated that our approach has potential in mite-infestation detection using UAV multispectral imagery.

We introduce a novel network, called CO-attention Siamese Network (COSNet), to address the unsupervised video object segmentation task from a holistic view. We emphasize the importance of inherent correlation among video frames and incorporate a global co-attention mechanism to improve further the state-of-the-art deep learning based solutions that primarily focus on learning discriminative foreground representations over appearance and motion in short-term temporal segments. The co-attention layers in our network provide efficient and competent stages for capturing global correlations and scene context by jointly computing and appending co-attention responses into a joint feature space. We train COSNet with pairs of video frames, which naturally augments training data and allows increased learning capacity. During the segmentation stage, the co-attention model encodes useful information by processing multiple reference frames together, which is leveraged to infer the frequently reappearing and salient foreground objects better. We propose a unified and end-to-end trainable framework where different co-attention variants can be derived for mining the rich context within videos. Our extensive experiments over three large benchmarks manifest that COSNet outperforms the current alternatives by a large margin.

The quality and accessibility of modern financial service have been quickly and dramatically improved, which benefits from the fast development of information technology. It has also witnessed the trend for applying artificial intelligence related technology, especially machine learning to the finance and security industry ranging from face recognition to fraud detection. In particular, deep neural networks have proven to be far superior to traditional algorithms in various application scenarios of computer vision. In this paper, we propose a deep learning-based video analysis system for automated compliance audit in stock brokerage, which in general consists of five modules here: 1) Video tampering and integrity detection; 2) Objects of interest localization and association; 3) Analysis of presence and departure of personnel in a video; 4) Face image quality assessment; and 5) Signature action positioning. To the best of our knowledge, this is the first work that introduces remote automated compliance audit system for the dual-recorded video in finance and security industry. The experimental results suggest our system can identify most of the potential non-compliant videos and has greatly improved the working efficiency of the auditors and reduced human labor costs. The collected dataset in our experiment will be released with this paper.

In this paper, we propose to use hypergraphs as the model for images and pose image segmentation as a machine learning problem in which some pixels (called seeds) are labeled as the objects and background. Using the seed pixels, our method predicts the labels for all unlabeled pixels. We present the relations of the proposed method to other hypergraph based learning techniques. We give an adaptive procedure for constructing image hypergraphs and achieve promising results on a real image dataset.

We propose a method for unsupervised video object segmentation by transferring the knowledge encapsulated in image-based instance embedding networks. The instance embedding network produces an embedding vector for each pixel that enables identifying all pixels belonging to the same object. Though trained on static images, the instance embeddings are stable over consecutive video frames, which allows us to link objects together over time. Thus, we adapt the instance networks trained on static images to video object segmentation and incorporate the embeddings with objectness and optical flow features, without model retraining or online fine-tuning. The proposed method outperforms state-of-the-art unsupervised segmentation methods in the DAVIS dataset and the FBMS dataset.

Tree species identification at the individual tree level is crucial for forest operations and management, yet its automated mapping remains challenging. Emerging technology, such as the high-resolution imagery from unmanned aerial vehicles (UAV) that is now becoming part of every forester’s surveillance kit, can potentially provide a solution to better characterize the tree canopy. To address this need, we have developed an approach based on a deep Convolutional Neural Network (CNN) to classify forest tree species at the individual tree-level that uses high-resolution RGB images acquired from a consumer-grade camera mounted on a UAV platform. This work explores the ability of the Dense Convolutional Network (DenseNet) to classify commonly available economic coniferous tree species in eastern Canada. The network was trained using multitemporal images captured under varying acquisition parameters to include seasonal, temporal, illumination, and angular variability. Validation of this ...

As an important problem in computer vision, salient object detection (SOD) from images has been attracting an increasing amount of research effort over the years. Recent advances in SOD, not surprisingly, are dominantly led by deep learning-based solutions (named deep SOD) and reflected by hundreds of published papers. To facilitate the in-depth understanding of deep SODs, in this paper we provide a comprehensive survey covering various aspects ranging from algorithm taxonomy to unsolved open issues. In particular, we first review deep SOD algorithms from different perspectives including network architecture, level of supervision, learning paradigm and object/instance level detection. Following that, we summarize existing SOD evaluation datasets and metrics. Then, we carefully compile a thorough benchmark results of SOD methods based on previous work, and provide detailed analysis of the comparison results. Moreover, we study the performance of SOD algorithms under different attributes, which have been barely explored previously, by constructing a novel SOD dataset with rich attribute annotations. We further analyze, for the first time in the field, the robustness and transferability of deep SOD models w.r.t. adversarial attacks. We also look into the influence of input perturbations, and the generalization and hardness of existing SOD datasets. Finally, we discuss several open issues and challenges of SOD, and point out possible research directions in future. All the saliency prediction maps, our constructed dataset with annotations, and codes for evaluation are made publicly available at https://github.com/wenguanwang/SODsurvey.

This paper proposes a deep learning model to efficiently detect salient regions in videos. It addresses two important issues: 1) deep video saliency model training with the absence of sufficiently large and pixel-wise annotated video data and 2) fast video saliency training and detection. The proposed deep video saliency network consists of two modules, for capturing the spatial and temporal saliency information, respectively. The dynamic saliency model, explicitly incorporating saliency estimates from the static saliency model, directly produces spatiotemporal saliency inference without time-consuming optical flow computation. We further propose a novel data augmentation technique that simulates video training data from existing annotated image data sets, which enables our network to learn diverse saliency information and prevents overfitting with the limited number of training videos. Leveraging our synthetic video data (150K video sequences) and real videos, our deep video saliency model successfully learns both spatial and temporal saliency cues, thus producing accurate spatiotemporal saliency estimate. We advance the state-of-the-art on the densely annotated video segmentation data set (MAE of .06) and the Freiburg-Berkeley Motion Segmentation data set (MAE of .07), and do so with much improved speed (2 fps with all steps).

One of the most important ecosystems in the Amazon rainforest is the Mauritia flexuosa swamp or “aguajal”. However, deforestation of its dominant species, the Mauritia flexuosa palm, also known as “aguaje”, is a common issue, and conservation is poorly monitored because of the difficult access to these swamps. The contribution of this paper is twofold: the presentation of a dataset called MauFlex, and the proposal of a segmentation and measurement method for areas covered in Mauritia flexuosa palms using high-resolution aerial images acquired by UAVs. The method performs a semantic segmentation of Mauritia flexuosa using an end-to-end trainable Convolutional Neural Network (CNN) based on the Deeplab v3+ architecture. Images were acquired under different environment and light conditions using three different RGB cameras. The MauFlex dataset was created from these images and it consists of 25,248 image patches of 512 × 512 pixels and their respective ground truth masks. The results ov...

We propose a light-weight variational framework for online tracking of object segmentations in videos based on optical flow and image boundaries. While high-end computer vision methods on this task rely on sequence specific training of dedicated CNN architectures, we show the potential of a variational model, based on generic video information from motion and color. Such cues are usually required for tasks such as robot navigation or grasp estimation. We leverage them directly for video object segmentation and thus provide accurate segmentations at potentially very low extra cost. Our simple method can provide competitive results compared to the costly CNN-based methods with parameter tuning. Furthermore, we show that our approach can be combined with state-of-the-art CNN-based segmentations in order to improve over their respective results. We evaluate our method on the datasets $\mathrm{DAVIS}_{16,17}$ and SegTrack v2.

Many states have limited amount of drinking water which becomes scarcer every year due to changing climate and growth. To manage their water resources wisely they encourage their residence to replace the grass on their loans with xeriscaping. Thus, they pay the residence to take of the grass which demands a great deal of water and replace it with gravel or desert plants demanding very little watering, or no more watering than the rainfall. The amount of money paying the residence to take of the grass and other water consuming vegetation is considerable often in the order of tens of thousands of dollars. Often, property owners take of the grass, take the money, and later on they change their mind or sell the property to another person and they reinstall the grass in the property. In such a case the authorities must take the money back. In order to automate the process of who maintains a loan with grass and who does not, an unmanned aerial vehicle with LIDAR is being used to automatic...

As an important problem in computer vision, salient object detection (SOD) from images has been attracting an increasing amount of research effort over the years. Recent advances in SOD, not surprisingly, are dominantly led by deep learning-based solutions (named deep SOD) and reflected by hundreds of published papers. To facilitate the in-depth understanding of deep SODs, in this paper we provide a comprehensive survey covering various aspects ranging from algorithm taxonomy to unsolved open issues. In particular, we first review deep SOD algorithms from different perspectives including network architecture, level of supervision, learning paradigm and object/instance level detection. Following that, we summarize existing SOD evaluation datasets and metrics. Then, we carefully compile a thorough benchmark results of SOD methods based on previous work, and provide detailed analysis of the comparison results. Moreover, we study the performance of SOD algorithms under different attributes, which have been barely explored previously, by constructing a novel SOD dataset with rich attribute annotations. We further analyze, for the first time in the field, the robustness and transferability of deep SOD models w.r.t. adversarial attacks. We also look into the influence of input perturbations, and the generalization and hardness of existing SOD datasets. Finally, we discuss several open issues and challenges of SOD, and point out possible research directions in future. All the saliency prediction maps, our constructed dataset with annotations, and codes for evaluation are made publicly available at https://github.com/wenguanwang/SODsurvey.

This paper addresses the problem of image segmentation by iterative region aggregations starting from an initial superpixel decomposition. Classical approaches for this task compute superpixel similarity using distance measures between superpixel descriptor vectors. This usually poses the well-known problem of the semantic gap and fails to properly aggregate visually non-homogeneous superpixels that belong to the same high-level object. This work proposes to use random forests to learn the merging probability between adjacent superpixels in order to overcome the aforementioned issues. Compared to existing works, this approach learns the fusion rules without explicit similarity measure computation. We also introduce a new superpixel context descriptor to strengthen the learned characteristics towards better similarity prediction. Image segmentation is then achieved by iteratively merging the most similar superpixel pairs selected using a similarity weighting objective function. Experimental results of our approach on four datasets including DAVIS 2017 and ISIC 2018 show its potential compared to state-of-the-art approaches.

In this paper, we present a novel Siamese graph convolution network (GCN) for face sketch recognition. To build a graph from an image, we utilize a deep learning method to detect the image edges, and then use a superpixel method to segment the edge image. Each segmented superpixel region is taken as a node, and each pair of adjacent regions forms an edge of the graph. Graphs from both a face sketch and a face photo are input into the Siamese GCN for recognition. A deep graph matching method is used to share messages between cross-modal graphs in this model. Experiments show that the GCN can obtain high performance on several face photo-sketch datasets, including seen and unseen face photo-sketch datasets. It is also shown that the model performance based on the graph structure representation of the data using the Siamese GCN is more stable than a Siamese CNN model.

Recent advances in visual tracking field design part-based model to handle the deformation and occlusion challenges. Previous methods only consider the sole degree of dependencies (e.g., pairwise or high-order dependencies) between object parts in consecutive frames. However, the degree of dependencies of different object parts in consecutive frames are not consistent, especially when large deformation and occlusion happen. To that end, we design a hybrid structure hypergraph based tracker, which use a non-uniform hypergraph to model the dependencies among object parts. The tracking task is further formulated as the dense structures extracting problem on the non-uniform hypergraph, which is solved by an approximate algorithm efficiently. Several experiments are carried out on publicly available online deformable object tracking dataset, i.e., Deform-SOT dataset, to demonstrate the favorable performance of the proposed method against the state-of-the-art online tracking methods.

Personal robots and driverless cars need to be able to operate in novel environments and thus quickly and efficiently learn to recognise new object classes. We address this problem by considering the task of video object segmentation. Previous accurate methods for this task finetune a model using the first annotated frame, and/or use additional inputs such as optical flow and complex post-processing. In contrast, we develop a fast, causal algorithm that requires no finetuning, auxiliary inputs or post-processing, and segments a variable number of objects in a single forward-pass. We represent an object with clusters, or "visual words", in the embedding space, which correspond to object parts in the image space. This allows us to robustly match to the reference objects throughout the video, because although the global appearance of an object changes as it undergoes occlusions and deformations, the appearance of more local parts may stay consistent. We learn these visual wor...

Pattern Recognition and Classification is considered one of the most promising applications in the scientific field of Artificial Neural Networks (ANN). However, regardless of the vast scientific advances in almost every aspect of the technology and mathematics, neural networks still need to be fairly large and complex (i.e., deep), in order to provide robust results. In this article, we propose a novel ANN architecture approach that aims to combine two fairly small Neural Networks based on an introduced probability term of correct classification. Additionally, we present a second ANN, used to reclassify the potentially incorrect results by using the most probable error-free results as additional training data with the predicted labels. The proposed method achieves a rapid decrease in the mean square error compared to other large and complex ANN architectures with a similar execution time. Our approach demonstrates increased effectiveness when applied to various databases, related t...

Object proposal detection is an effective way of accelerating object recognition. Existing proposal methods are mostly based on detecting object boundaries, which may not be effective for cluttered backgrounds. In this paper, we leverage stereopsis as a robust and effective solution for generating object proposals. We first obtain a set of candidate bounding boxes through adaptive transformation, which fits the bounding boxes tightly to object boundaries detected by rough depth and color information. A two-level hierarchy composed of proposal and cluster levels is then constructed to estimate object locations in an efficient and accurate manner. Three stereo-based cues “exactness,” “focus,” and “distribution” are proposed for objectness estimation. Two-level hierarchical ranking is proposed to accurately obtain ranked object proposals. A stereo data set with 400 labeled stereo image pairs is constructed to evaluate the performance of the proposed method in both indoor and outdoor sc...

Reliable object discovery in realistic indoor scenes is a necessity for many computer vision and service robot applications. In these scenes, semantic segmentation methods have made huge advances in recent years. Such methods can provide useful prior information for object discovery by removing false positives and by delineating object boundaries. We propose a novel method that combines bottom-up object discovery and semantic priors for producing generic object candidates in RGB-D images. We use a deep learning method for semantic segmentation to classify colour and depth superpixels into meaningful categories. Separately for each category, we use saliency to estimate the location and scale of objects, and superpixels to find their precise boundaries. Finally, object candidates of all categories are combined and ranked. We evaluate our approach on the NYU Depth V2 dataset and show that we outperform other state-of-the-art object discovery methods in terms of recall.

Accurately mapping individual tree species in densely forested environments is crucial to forest inventory. When considering only RGB images, this is a challenging task for many automatic photogrammetry processes. The main reason for that is the spectral similarity between species in RGB scenes, which can be a hindrance for most automatic methods. This paper presents a deep learning-based approach to detect an important multi-use species of palm trees (Mauritia flexuosa; i.e., Buriti) on aerial RGB imagery. In South-America, this palm tree is essential for many indigenous and local communities because of its characteristics. The species is also a valuable indicator of water resources, which comes as a benefit for mapping its location. The method is based on a Convolutional Neural Network (CNN) to identify and geolocate singular tree species in a high-complexity forest environment. The results returned a mean absolute error (MAE) of 0.75 trees and an F1-measure of 86.9%. These result...

Tree species identification at the individual tree level is crucial for forest operations and management, yet its automated mapping remains challenging. Emerging technology, such as the high-resolution imagery from unmanned aerial vehicles (UAV) that is now becoming part of every forester’s surveillance kit, can potentially provide a solution to better characterize the tree canopy. To address this need, we have developed an approach based on a deep Convolutional Neural Network (CNN) to classify forest tree species at the individual tree-level that uses high-resolution RGB images acquired from a consumer-grade camera mounted on a UAV platform. This work explores the ability of the Dense Convolutional Network (DenseNet) to classify commonly available economic coniferous tree species in eastern Canada. The network was trained using multitemporal images captured under varying acquisition parameters to include seasonal, temporal, illumination, and angular variability. Validation of this ...

Genetic syndromes develop different symptoms. One of the commonly observed symptoms is facial dysmorphisms. Hence syndromes may be detected based on facial images. This paper proposes a Radon transform based method for detection of facial dysmorphism. Detection results were improved using bit plane slice approach. Most significant four bit planes of Radon transformed image are proposed to be used for feature extraction. Linear Discriminant Classifier (LDC) and Quadratic Discriminant classifier (QDC) were used for classification. Comparing the results of Radon transform and Radon transform and bit plane slicing, it is observed that accuracy is more by 2.71% using QDC and 10.81% using LDC in later case.

Mapping of tree crowns is important in agriculture and ecology. Current manual systems of tree crown mapping are cumbersome and inefficient. In recent years, UAVs are emerging as a platform for remote sensing that complements traditional satellite based remote sensing systems. Remote sensing from a UAV also known as Low Altitude Remote Sensing (LARS) provides interesting options for agriculture since they allow study of crops at a sub decimeter ground separation distance (GSD). We propose Banana-Seg fully Convolutional deep neural network architecture for mapping of tree crowns in aerial imagery. Banana-Seg is a two dimensional Convolutional Neural Network (2D-CNN) architecture. The performance of tree crown mapping using Banana-Seg is evaluated by performance parameters derived from a confusion matrix or contingency matrix. Precision, recall, accuracy and F1- score performance are evaluated. The performance of Banana-Seg method is compared with a one dimensional CNN (1D-CNN) architecture. Further, visualization of performance is presented for several test images. The results indicate that the proposed Banana-Seg architecture outperforms the 1D-CNN method

There have been times when we’re outside, having fast food, and we wonder about the food that we’re having. Is it healthy, what is its caloric content, how much protein, fat it contains? So we decided to create an AI/ML model, train it on images of 10 fast foods, in the hope that it correctly identifies whatever fast food is thrown at it, along with the nutrition details of the respective food item.

There have been times when we’re outside, having fast food, and we wonder about the food that we’re having. Is it healthy, what is its caloric content, how much protein, fat it contains? So we decided to create an AI/ML model, train it on images of 10 fast foods, in the hope that it correctly identifies whatever fast food is thrown at it, along with the nutrition details of the respective food item.

Deep Learning(DL) is one of the parts of machine learning methods based on artificial intelligence network with the representation of learning. Deep learning techniques help to process and analyze big data available around us through several applications in various fields related to the subject. The concept of plant disease is the scientific study of plants where the disease in plants is caused by pathogens and other environmental conditions. In order to identify the disease and curing them in the initial stage is the better option. Automatic and perfect identification of plant disease is important in the aspects of food security, managing disease, and predicting it. DL method helps in detecting the disease's severe-ness. With the leaf of apple plant with rot images in the Plant-Village dataset are explained by botanists with 4 stages of severeness. The deep learning convolutional neural network are trained for analysing the severe-ness in the plant disease.
This research tries to analyze the transfer learning method with the help of a deep model and trained networks from scratch. The deep VGG16 model under the training of transfer learning was found to have an accuracy of 90 percent on the test. The DL method will have a huge significance in disease control in plants in the field of modern agriculture.

With the target of simultaneously segmenting semantically related videos to identify the common objects, video object cosegmentation has attracted the attention of researchers in recent years. Existing methods are primarily based on pair-wise relations between adjacent pixels and regions, which are susceptible to performance degradation from object entries/exists or occlusions. Specifically, we refer these video frames without the common objects present as the "empty" frames. In this paper, we propose a multilevel hypergraph-based full Video object CoSegmentation (VCS) method, which incorporates high-level semantics and low-level appearance/motion/saliency to construct the hyperedge among multiple spatially and temporally adjacent regions. Specifically, the high-level semantic model fuses multiple object proposals from each frame instead of relying on a single object proposal per frame. A hypergraph cut is subsequently utilized to calculate the object cosegmentation. Experiments on four video object segmentation/cosegmentation datasets against state-of-the-art methods with both objective and subjective results manifest the effectiveness of the proposed VCS method, including the SegTrack and VCoSeg datasets without "empty" frames, the XJTU-Stevens dataset with 3.7% "empty" frames, and the Noisy-ViCoSeg dataset proposed together with our method with 30.3% "empty" frames.

Defined as simultaneously segmenting a set of related videos to identify the common objects, video co-segmentation has attracted the attention of researchers in recent years. Existing methods are primarily based on pair-wise relations between adjacent pixels/regions, which are susceptible to performance degradation from “empty” video frames (e.g., due to transient/intermittent common objects). In this paper, a new multilevel hypergraph based method, termed the full Video object Co-Segmentation method (VCS), is proposed, which incorporates both a high-level semantics object model and a low-level appearance/motion/saliency object model to construct the hyperedge among multiple spatially and temporally adjacent regions. Specifically, the high-level semantic model fuses multiple object proposals from each frame instead of relying on a single object proposal per frame. A hypergraph cut is subsequently utilized to calculate the object co-segmentation. Experiments on three datasets demonstrate the efficacy of the proposed VCS method.

Damage to buildings such as crack and exfoliation requires continuous maintenance because the risk of safety accidents increases. However, existing detection method for the damage is visual inspection method, there is a limitation. Recently, unmanned aerial vehicle with high mobility and versatility equipped with various sensors is attracting attention in safety inspection. This study detected and quantified the crack and exfoliation appearing on the exterior of a building using an unmanned aerial vehicle. As a result of the study, 4 exfoliations and 8 cracks were detected. And the detected cracks were quantified using image processing techniques. This study is expected to be used as a basic study when developing building safety inspection techniques using an unmanned aerial vehicle.

Intertidal zone is not only an important marine ecosystem but also serves as a buffer zone to minimize the impact of natural disasters such as storms and coastal erosion on the interior land. To study the intertidal zone, the topographic map of the intertidal zone is of great importance. However, with the specific geographical location, the establishment of topographic maps of intertidal zone is very difficult if applying traditional surveying methods. Unmanned aerial vehicle with low flight altitude, flexible flight ability can overcome the limitations of traditional measurement methods in making the intertidal topographic map. In this paper, we present the approach of using unmanned aerial vehicle in intertidal topographic mapping. Our method focuses on selecting the optimal timing for flight to determine the extent and topography of the intertidal zone. The orientation element of the image is determined by satellite positioning technology combined with the IMU system in order to overcome the limitation of the ground control arrangement in intertidal zone. We then evaluated the results of the intertidal topographic mapping using drone technology.

Live fish recognition is a difficult multi-class order task in the open sea. We propose a technique to perceive fish in an unlimited common habitat.In the proposed technique, VGG-16 with deep fish architecture is used to enhance the feature extraction what's more, to improve the exactness of the result.The proposed approach comprises of two fundamental stages; namely Fish Localization phase and Fish classification phase.The technique first detect the fish from the image by extracting feature map using VGG16 network. DeepFish architectureis used to categorize the Fish.Then, the proposed approach uses support vector machine and random forest classifier to differentiate between fish species. Experimental results obtained show that VGG16 with deepfish architecture using support vector machine attains a better accuracy of 99.47%.

In this paper, the proposed image style transfer methodology using the Convolutional neural network, given a random pair of images, a universal image style transfer technique extract, the image texture from a reference image to synthesize an output supported the design of a content image. Image processing algorithms supported second-order statistics however are either computationally high-priced or vulnerable to generate artifacts because of the trade-off between image quality and run-time performance. Recently there has been much progress within the field of image style transfer, a process that aims at redrawing an image within the type of another image. During this paper, the proposed technique consists of a normalization step and a smoothing step. Whereas the stylization step transfers the design of the reference image to the content photograph, the smoothing step ensures spatially consistent stylizations. Every one of the steps includes a closed-form solution and maybe computed efficiently. This paper tends to conduct extensive experimental validations. The results show that the proposed technique generates photorealistic stylization outputs that are additional most popular by human subjects as compared to those by the competitive strategies, whereas, running a lot of faster.