Occlusion-robust Detector Trained with Occluded Pedestrians

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Pedestrian detection relying on deep convolution neural networks has made significant progress. Though promising results have been achieved on standard pedestrians, the performance on heavily occluded pedestrians remains far from satisfactory. The main culprits are intra-class occlusions involving other pedestrians and inter-class occlusions caused by other objects, such as cars and bicycles. These result in a multitude of occlusion patterns. We propose an approach for occluded pedestrian detection with the following contributions. First, we introduce a novel maskguided attention network that fits naturally into popular pedestrian detection pipelines. Our attention network emphasizes on visible pedestrian regions while suppressing the occluded ones by modulating full body features. Second, we empirically demonstrate that coarse-level segmentation annotations provide reasonable approximation to their dense pixel-wise counterparts. Experiments are performed on CityPersons and Caltech datasets. Our approach sets a new state-of-the-art on both datasets. Our approach obtains an absolute gain of 9.5% in log-average miss rate, compared to the best reported results [31] on the heavily occluded HO pedestrian set of CityPersons test set. Further, on the HO pedestrian set of Caltech dataset, our method achieves an absolute gain of 5.0% in log-average miss rate, compared to the best reported results [13]. Code and models are available at: https://github.com/Leotju/MGAN .

International Journal of Electrical and Computer Engineering (IJECE), 2023

The investigation of a deep neural network for pedestrian classification using transfer learning methods is proposed in this study. The development of deep convolutional neural networks has significantly improved the autonomous driver assistance system for pedestrian classification. However, the presence of partially occluded parts and the appearance variation under complex scenes are still robust to challenge in the pedestrian detection system. To address this problem, we proposed six transfer learning models: end-to-end convolutional neural network (CNN) model, scratch-trained residual network (ResNet50) model, and four transfer learning models: visual geometry group 16 (VGG16), GoogLeNet (InceptionV3), ResNet50, and MobileNet. The performance of the pedestrian classification was evaluated using four publicly datasets: Institut National de Recherche en Sciences et Technologies du Numérique (INRIA), Prince of Songkla University (PSU), CVC05, and Walailak University (WU) datasets. The experimental results show that six transfer learning models achieve classification accuracy of 65.2% (end-to-end CNN), 92.92% (scratch-trained ResNet50), 97.15% (pre-trained VGG16), 94.39% (pre-trained InceptionV3), 90.43% (pre-trained ResNet50), and 98.69% (pre-trained MobileNet) using data from Southern Thailand (PSU dataset). Further analysis reveals that the deeper the ConvNet architecture, the more specific information of features is provided. In addition, the deep ConvNet architecture can distinguish pedestrian occluded patterns while being trained with partially occluded parts of data samples.

Advances in Multimedia

Pedestrian detection and classification are of increased interest in the intelligent transportation system (ITS), and among the challenging issues, we can find limitations of tiny and occluded appearances, large variation of human pose, cluttered background, and complex environment. In fact, a partial occlusion handling is important in the case of detecting pedestrians, in order to avoid accidents between pedestrians and vehicles, since it is difficult to detect when pedestrians are suddenly crossing the road. To solve the partial occlusion problem, a pyramidal part-based model (PPM) is proposed to obtain a more accurate prediction based on the majority vote of the confidence score of the visible parts by cascading the pyramidal structure. The experimental results on the proposed scheme achieved 96.25% accuracy on the INRIA dataset and 81% accuracy on the PSU (Prince of Songkla University) dataset. Our proposed model can be applied in the real-world environment to classify the occlu...

arXiv (Cornell University), 2022

Robust detection of vulnerable road users is a safety critical requirement for the deployment of autonomous vehicles in heterogeneous traffic. One of the most complex outstanding challenges is that of partial occlusion where a target object is only partially available to the sensor due to obstruction by another foreground object. A number of leading pedestrian detection benchmarks provide annotation for partial occlusion, however each benchmark varies greatly in their definition of the occurrence and severity of occlusion. Recent research demonstrates that a high degree of subjectivity is used to classify occlusion level in these cases and occlusion is typically categorized into 2-3 broad categories such as "partially" and "heavily" occluded. This can lead to inaccurate or inconsistent reporting of pedestrian detection model performance depending on which benchmark is used. This research introduces a novel, objective benchmark for partially occluded pedestrian detection to facilitate the objective characterization of pedestrian detection models. Characterization is carried out on seven popular pedestrian detection models for a range of occlusion levels from 0-99%. Results demonstrate that pedestrian detection performance degrades, and the number of false negative detections increase as pedestrian occlusion level increases. Of the seven popular pedestrian detection routines characterized, CenterNet has the greatest overall performance, followed by SSDlite. Reti-naNet has the lowest overall detection performance across the range of occlusion levels.

2007

Abstract In this paper we present a simple approach for person detection in surveillance for static cameras. The basic idea is to train a separate classifier for each image location which has only to discriminate the object from the background at a specific location. This is a considerably simpler problem than the detection of persons on arbitrary backgrounds. Therefore, we use adaptive classifiers which are trained online. Due to the reduced complexity we can use a simple update strategy that requires only a few positive samples ...

2007

The identification of pedestrians is an important problem in a wide range of computer vision applications. Previous work on pedestrian identification mainly focuses on feature extraction approaches. In this study, we investigate four new features for pedestrian identification that aim to overcome the limitations of features that are employed in previous studies. In particular, we focus on features that are to some extent robust to the presence of occlusions in the pedestrian images. The four new feature extraction approaches that are investigated are based on: (1) nonlinear dimensionality reduction, (2) texton frequency histograms, (3) Latent Dirichlet Allocation, and (4) local receptive fields networks. We test the performance of our feature extraction approaches on the NiSIS competition dataset. From the obtained results, we conclude that Support Vector Machines trained on PCA features perform best, most likely due to the exploitation of biases in the NiSIS dataset.

2021

Pedestrian detection is used in many vision based applications ranging from video surveillance to autonomous driving. Despite achieving high performance, it is still largely unknown how well existing detectors generalize to unseen data. This is important because a practical detector should be ready to use in various scenarios in applications. To this end, we conduct a comprehensive study in this paper, using a general principle of direct cross-dataset evaluation. Through this study, we find that existing state-of-the-art pedestrian detectors, though perform quite well when trained and tested on the same dataset, generalize poorly in cross dataset evaluation. We demonstrate that there are two reasons for this trend. Firstly, their designs (e.g. anchor settings) may be biased towards popular benchmarks in the traditional single-dataset training and test pipeline, but as a result largely limit their generalization capability. Secondly, the training source is generally not dense in pede...

Applied Sciences, 2021

Although many deep-learning-based methods have achieved considerable detection performance for pedestrians with high visibility, their overall performances are still far from satisfactory, especially when heavily occluded instances are included. In this research, we have developed a novel pedestrian detector using a deformable attention-guided network (DAGN). Considering that pedestrians may be deformed with occlusions or under diverse poses, we have designed a deformable convolution with an attention module (DCAM) to sample from non-rigid locations, and obtained the attention feature map by aggregating global context information. Furthermore, the loss function was optimized to get accurate detection bounding boxes, by adopting complete-IoU loss for regression, and the distance IoU-NMS was used to refine the predicted boxes. Finally, a preprocessing technique based on tone mapping was applied to cope with the low visibility cases due to poor illumination. Extensive evaluations were ...

IEEE

Pedestrian detection is an important area in com­ puter vision with key applications in intelligent vehicle and surveillance systems. One of the main challenges in pedestrian detection is occlusion. In this paper, we propose a novel pedestrian detection approach capable of handling partial occlusion. Three stage cascaded classifier is used in the proposed approach. Global classifier based on HOG features and linear-SVM is first employed to classify the whole scanning window. For ambiguous patterns, a set of part-based classifiers trained on features derived from non-occluded dataset are employed on the second stage. Several fusion methods including average, maximum, linear and non-linear SVM classifiers are examined to combine the obtained part scores. The linear/non-linear fusion coefficients are estimated by learning an additional third stage SVM classifier. The training data in the third stage classifier is augmented by generating a set of artificially occluded samples which simulate real occlusion conditions commonly occurred in pedestrians. Experimental results using Daimler and INRIA data sets show the effectiveness of the proposed approach.

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020

Pedestrian detection methods have been significantly improved with the development of deep convolutional neural networks. Nevertheless, robustly detecting pedestrians with a large variant on sizes and with occlusions remains a challenging problem. In this paper, we propose a gated multi-layer convolutional feature extraction method which can adaptively generate discriminative features for candidate pedestrian regions. The proposed gated feature extraction framework consists of squeeze units, gate units and a concatenation layer which perform feature dimension squeezing, feature elements manipulation and convolutional features combination from multiple CNN layers, respectively. We proposed two different gate models which can manipulate the regional feature maps in a channel-wise selection manner and a spatial-wise selection manner, respectively. Experiments on the challenging CityPersons dataset demonstrate the effectiveness of the proposed method, especially on detecting those small-size and occluded pedestrians.