Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Work
Depth Camera and Data Pre-Processing
Results
Pandora Dataset
Watch-N-Patch Dataset
Experimental Results
Conclusions
References
All Topics
Computer Science
Computer Vision

Head Detection with Depth Images in the Wild

Profile image of Guido BorghiGuido Borghi

2018, Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications

https://doi.org/10.5220/0006541000560063
visibility

…

description

9 pages

link

1 file

Abstract

Head detection and localization is a demanding task and a key element for many computer vision applications, like video surveillance, Human Computer Interaction and face analysis. The stunning amount of work done for detecting faces on RGB images, together with the availability of huge face datasets, allowed to setup very effective systems on that domain. However, due to illumination issues, infrared or depth cameras may be required in real applications. In this paper, we introduce a novel method for head detection on depth images that exploits the classification ability of deep learning approaches. In addition to reduce the dependency on the external illumination, depth images implicitly embed useful information to deal with the scale of the target objects. Two public datasets have been exploited: the first one, called Pandora, is used to train a deep binary classifier with face and non-face images. The second one, collected by Cornell University, is used to perform a cross-dataset test during daily activities in unconstrained environments. Experimental results show that the proposed method overcomes the performance of state-of-art methods working on depth images.

Related papers

Human Face Detection with and without masks using Deep-learning
IRJET Journal

IRJET, 2022

The worldwide health system is being caused havoc by the COVID-19 (Coronavirus) pandemic. According to WHO, wearing a mask in public places is one of the most important barriers or protections for reducing virus transmission. Wearing a facemask will reduce the possibility of virus transmission. Thus, we need physical monitoring to ensure that everyone is wearing a facemask, which is unsafe in this situation. Our key goal is to use deep learning and determine whether or not the person in the picture or video stream is wearing a mask. As a result, therefore, with the help of this project, a person in charge of monitoring individuals will be able to control and give instructions from a distant place. This paper discusses the detection of human face with and without masks by using various libraries of python such as Keras, TensorFlow, open-cv and by training the dataset using MobileNetV2 architecture. It has one residual block with stride 1 and a stride 2 downsizing block with a ReLU6 as an activation function for the hidden layers and SoftMax for the output layer. We have finetuned Mobile-Net by adding a Fully Connected (FC) layer to the base model and achieved a testing accuracy of 99.28%. Using real-time videos, the trained model is tested in a variety of scenarios and is adapted and modified accurately. This study will help in preventing virus transmission and reducing contact with the virus.

View PDFchevron_right
Robust Head Detection in Complex Videos Using Two-Stage Deep Convolution Framework
Yasir Ali

IEEE Access

Pedestrian head detection plays an important role in identifying and localizing individuals in real world visual data. Head detection is a nontrivial problem due to considerable variance in camera viewpoints, scales, human poses, and appearances in the scene. Thanks to the translation invariance property of convolutional neural networks (CNNs) which enables large capacity CNNs to handle the problem of appearance and pose variations in the scene. However, the problem of scale invariance is still an open issue. To address this problem, this paper presents a two-stage head detection framework that utilizes fully convolutional network (FCN) to generate scale-aware proposals followed by CNN that classifies each proposal into two classes, i.e. head and background. Experiments results show that using scale-aware proposals obtained by FCN, the object recall rate and mean average precision (mAP) are improved. Additionaly, we demonstrate that our framework achieved state-of-the-art results on four challenging benchmark datasets, i.e. HollywoodHeads, Casablanca, SHOCK, and WIDERFACE.

View PDFchevron_right
Deep Learning Fusion of RGB and Depth Images for Pedestrian Detection
Peter Veelaert

2019

In this paper, we propose an effective method based on the Faster-RCNN structureto combine RGB and depth images for pedestrian detection. During the training stage,we generate a semantic segmentation map from the depth image and use it to refine theconvolutional features extracted from the RGB images. In addition, we acquire moreaccurate region proposals by exploring the perspective projection with the help of depthinformation. Experimental results demonstrate that our proposed method achieves thestate-of-the-art RGBD pedestrian detection performance on KITTI [12] datase

View PDFchevron_right
Utilizing YOLO 8x Models, Deep Learning-Based Head protector Detection for Construction Workers
CHITHIK RAJA MOHAMED SINNAIYA

Many organizations in the sector are becoming increasingly concerned about the safety of construction workers on project sites. At construction sites, safety helmets can help reduce worker injuries, but for a variety of reasons, they are frequently not worn correctly. Therefore, it is essential to have an autonomous safety helmet detection system that uses computer vision. The You Only Look Once (YOLO) principle of real-time computer vision is the foundation of this research poster. After power-law transformation, image rescaling was utilized as the initial stage in image improvement. The YOLOv8 object detection approach was used in the design of a computer vision system to determine whether or not employees were wearing helmets. Our research showed that the YOLOv8x architecture's 95.46% mean average precision (mAP) produced great results in identifying objects. Our experimental findings revealed that 95.46% of the mean average precision (mAP) in YOLOv8x architecture demonstrate...

View PDFchevron_right
Improved Face Detection Method via Learning Small Faces on Hard Images Based on a Deep Learning Approach
Akmalbek Abdusalomov

Sensors

Most facial recognition and face analysis systems start with facial detection. Early techniques, such as Haar cascades and histograms of directed gradients, mainly rely on features that had been manually developed from particular images. However, these techniques are unable to correctly synthesize images taken in untamed situations. However, deep learning’s quick development in computer vision has also sped up the development of a number of deep learning-based face detection frameworks, many of which have significantly improved accuracy in recent years. When detecting faces in face detection software, the difficulty of detecting small, scale, position, occlusion, blurring, and partially occluded faces in uncontrolled conditions is one of the problems of face identification that has been explored for many years but has not yet been entirely resolved. In this paper, we propose Retina net baseline, a single-stage face detector, to handle the challenging face detection problem. We made ...

View PDFchevron_right
SAFETY HELMET DETECTION IN INDUSTRIAL ENVIRONMENT USING DEEP LEARNING
Computer Science & Information Technology (CS & IT) Computer Science Conference Proceedings (CSCP)

Safety is of predominant value for employees who are working in an industrial and construction environment. Real time Object detection is an important technique to detect violations of safety compliance in an industrial setup. The negligence in wearing safety helmets could be hazardous to workers, hence the requirement of the automatic surveillance system to detect persons not wearing helmets is of utmost importance and this would reduce the labor-intensive work to monitor the violations. In this paper, we deployed an advanced Convolutional Neural Network (CNN) algorithm called Single Shot Multibox Detector (SSD) to monitor violations of safety helmets. Various image processing techniques are applied to all the video data collected from the industrial plant. The practical and novel safety detection framework is proposed in which the CNN first detects persons from the video data and in the second step it detects whether the person is wearing the safety helmet. Using the proposed model, the deep learning inference benchmarking is done with Dell Advanced Tower workstation. The comparative study of the proposed approach is analysed in terms of detection accuracy (average precision) which illustrates the effectiveness of the proposed framework.

View PDFchevron_right
3D Head Pose Estimation through Facial Features and Deep Convolutional Neural Networks
Jehad Ali

Computers, Materials & Continua, 2021

Face image analysis is one among several important cues in computer vision. Over the last five decades, methods for face analysis have received immense attention due to large scale applications in various face analysis tasks. Face parsing strongly benefits various human face image analysis tasks inducing face pose estimation. In this paper we propose a 3D head pose estimation framework developed through a prior end to end deep face parsing model. We have developed an end to end face parts segmentation framework through deep convolutional neural networks (DCNNs). For training a deep face parts parsing model, we label face images for seven different classes, including eyes, brows, nose, hair, mouth, skin, and back. We extract features from gray scale images by using DCNNs. We train a classifier using the extracted features. We use the probabilistic classification method to produce gray scale images in the form of probability maps for each dense semantic class. We use a next stage of DCNNs and extract features from grayscale images created as probability maps during the segmentation phase. We assess the performance of our newly proposed model on four standard head pose datasets, including Pointing'04, Annotated Facial Landmarks in the Wild (AFLW), Boston University (BU), and ICT-3DHP, obtaining superior results as compared to previous results.

View PDFchevron_right
Object Detection with Deep Learning: A Review
yuelou fang

Due to object detection's close relationship with video analysis and image understanding, it has attracted much research attention in recent years. Traditional object detection methods are built on handcrafted features and shallow trainable architectures. Their performance easily stagnates by constructing complex ensembles which combine multiple low-level image features with high-level context from object detectors and scene classifiers. With the rapid development in deep learning, more powerful tools, which are able to learn semantic, high-level, deeper features, are introduced to address the problems existing in traditional architectures. These models behave differently in network architecture, training strategy and optimization function, etc. In this paper, we provide a review on deep learning based object detection frameworks. Our review begins with a brief introduction on the history of deep learning and its representative tool, namely Convolutional Neural Network (CNN). Then we focus on typical generic object detection architectures along with some modifications and useful tricks to improve detection performance further. As distinct specific detection tasks exhibit different characteristics, we also briefly survey several specific tasks, including salient object detection, face detection and pedestrian detection. Experimental analyses are also provided to compare various methods and draw some meaningful conclusions. Finally, several promising directions and tasks are provided to serve as guidelines for future work in both object detection and relevant neural network based learning systems.

View PDFchevron_right
An Efficient Safety and Authorized Helmet Detection Using Deep Learning Approach
Bernard Fabro

2022 International Conference on Smart Information Systems and Technologies (SIST), 2022

In recent years, riding a motorcycle has become one of the most convenient ways for consumers to go to their destination. Helmets are very important and necessary for the safety of motorcyclists, however, officers find it difficult to enforce the laws regarding the wearing of authorized helmets. This system was established to determine whether or not a helmet is allowed. The authorized helmet for riders is the full-face helmet. This vision-based system could aid in identifying riders who are wearing helmets that aren't fitting properly. The system used the YOLOv3 model and a deep learning approach to use the helmet detection model in this work. As a result, the model with the best training received an mAP of 97%. This proposed device may be used on busy routes to monitor motorcyclists to see if they are wearing an authorized helmet. Keywords—deep learning; full-face helmet; helmet detection; vison-based system; YOLOv3

View PDFchevron_right
Object Detection Techniques based on Deep Learning: A Review
Utkarsh Namdev

Computer Science & Engineering: An International Journal, 2022

Object detection is a computer technique that searches digital images and videos for occurrences of meaningful subjects in particular categories (such as people, buildings, and automobiles). It is related to computer vision and image processing. Two well-studied aspects of identification are facial and pedestrian detection. Object detection is useful in a wide range of visual recognition tasks, including image retrieval and video monitoring. The object detection algorithm has been improved many times to improve the performance in terms of speed and accuracy. “Due to the tireless efforts of many researchers, deep learning algorithms are rapidly improving their object detection performance. Pedestrian detection, medical imaging, robotics, self-driving cars, face recognition and other popular applications have reduced labor in many areas.” It is used in a wide variety of industries, with applications range from individual safeguarding to business productivity. It is a fundamental compo...

View PDFchevron_right

References (32)

  1. Bagdanov, A. D., Del Bimbo, A., and Masi, I. (2011). The florence 2d/3d hybrid face dataset. In Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding, pages 79-80. ACM.
  2. Baltrušaitis, T., Robinson, P., and Morency, L.-P. (2012). 3d constrained local model for rigid and non-rigid facial tracking. In Computer Vision and Pattern Recogni- tion (CVPR), 2012 IEEE Conference on, pages 2610- 2617. IEEE.
  3. Borghi, G., Gasparini, R., Vezzani, R., and Cucchiara, R. (2017a). Embedded recurrent network for head pose estimation in car. In Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV).
  4. Borghi, G., Venturelli, M., Vezzani, R., and Cucchiara, R. (2017b). Poseidon: Face-from-depth for driver pose estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  5. Bourdev, L. and Malik, J. (2009). Poselets: Body part de- tectors trained using 3d human pose annotations. In Computer Vision, 2009 IEEE 12th International Con- ference on, pages 1365-1372. IEEE.
  6. Chen, S., Bremond, F., Nguyen, H., and Thomas, H. (2016). Exploring depth information for head detection with depth images. In Advanced Video and Signal Based Surveillance (AVSS), 2016 13th IEEE International Conference on, pages 228-234. IEEE.
  7. Chollet, F. et al. (2015). Keras.
  8. Cortes, C. and Vapnik, V. (1995). Support vector machine. Machine learning, 20(3):273-297.
  9. Dalal, N. and Triggs, B. (2005). Histograms of oriented gra- dients for human detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Com- puter Society Conference on, volume 1, pages 886- 893. IEEE.
  10. Fanelli, G., Weise, T., Gall, J., and Van Gool, L. (2011). Real time head pose estimation from consumer depth cameras. In Joint Pattern Recognition Symposium, pages 101-110. Springer.
  11. Freund, Y. and Schapire, R. E. (1995). A desicion-theoretic generalization of on-line learning and an application to boosting. In European conference on computa- tional learning theory, pages 23-37. Springer.
  12. Frigieri, E., Borghi, G., Vezzani, R., and Cucchiara, R. (2017). Fast and accurate facial landmark localization in depth images for in-car applications. In Proceed- ings of the 19th International Conference on Image Analysis and Processing (ICIAP).
  13. Ikemura, S. and Fujiyoshi, H. (2011). Real-time human detection using relational depth similarity features. Computer Vision-ACCV 2010, pages 25-38.
  14. Khan, M. H., Shirahama, K., Farid, M. S., and Grzegorzek, M. (2016). Multiple human detection in depth images. In Multimedia Signal Processing (MMSP), 2016 IEEE 18th International Workshop on, pages 1-6. IEEE.
  15. Kingma, D. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
  16. Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012). Im- agenet classification with deep convolutional neural networks. In Advances in neural information process- ing systems, pages 1097-1105.
  17. Levi, K. and Weiss, Y. (2004). Learning object detection from a small number of examples: the importance of good features. In Computer Vision and Pattern Recog- nition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, volume 2, pages II-II. IEEE.
  18. Lowe, D. G. (1999). Object recognition from local scale- invariant features. In Computer vision, 1999. The pro- ceedings of the seventh IEEE international conference on, volume 2, pages 1150-1157. Ieee.
  19. Nghiem, A. T., Auvinet, E., and Meunier, J. (2012). Head detection using kinect camera and its application to fall detection. In Information Science, Signal Process- ing and their Applications (ISSPA), 2012 11th Inter- national Conference on, pages 164-169. IEEE.
  20. Osuna, E., Freund, R., and Girosit, F. (1997). Training support vector machines: an application to face de- tection. In Computer vision and pattern recognition, 1997. Proceedings., 1997 IEEE computer society con- ference on, pages 130-136. IEEE.
  21. Rowley, H. A., Baluja, S., and Kanade, T. (1998). Neural network-based face detection. IEEE Transactions on pattern analysis and machine intelligence, 20(1):23- 38.
  22. Sarbolandi, H., Lefloch, D., and Kolb, A. (2015). Kinect range sensing: Structured-light versus time-of-flight kinect. Computer vision and image understanding, 139:1-20.
  23. Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finoc- chio, M., Blake, A., Cook, M., and Moore, R. (2013). Real-time human pose recognition in parts from sin- gle depth images. Communications of the ACM, 56(1):116-124.
  24. Theano Development Team (2016). Theano: A Python framework for fast computation of mathematical ex- pressions. arXiv e-prints, abs/1605.02688.
  25. Venturelli, M., Borghi, G., Vezzani, R., and Cucchiara, R. (2016). Deep head pose estimation from depth data for in-car automotive applications. In Proceedings of the 2nd International Workshop on Understanding Hu- man Activities through 3D Sensors, ICPR workshop.
  26. Venturelli, M., Borghi, G., Vezzani, R., and Cucchiara, R. (2017). From depth data to head pose estimation: a siamese approach. In Proceedings of the 12th Inter- national Joint Conference on Computer Vision, Imag- ing and Computer Graphics Theory and Applications (VISAPP).
  27. Viola, P. and Jones, M. J. (2004). Robust real-time face detection. International journal of computer vision, 57(2):137-154.
  28. Vu, T.-H., Osokin, A., and Laptev, I. (2015). Context-aware cnns for person head detection. In Proceedings of the IEEE International Conference on Computer Vision, pages 2893-2901.
  29. Wu, B. and Nevatia, R. (2005). Detection of multiple, par- tially occluded humans in a single image by bayesian combination of edgelet part detectors. In Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on, volume 1, pages 90-97. IEEE.
  30. Wu, C., Zhang, J., Savarese, S., and Saxena, A. (2015). Watch-n-patch: Unsupervised understanding of ac- tions and relations. In The IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR).
  31. Xia, L., Chen, C.-C., and Aggarwal, J. K. (2011). Hu- man detection using depth information by kinect. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2011 IEEE Computer Society Conference on, pages 15-22. IEEE.
  32. Zhu, X. and Ramanan, D. (2012). Face detection, pose es- timation, and landmark localization in the wild. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 2879-2886. IEEE.

Related papers

Fully Convolutional Network for Head Detection with Depth Images
Guido Borghi

2018 24th International Conference on Pattern Recognition (ICPR), 2018

Head detection and localization are one of the most investigated and demanding tasks of the Computer Vision community. These are also a key element for many disciplines, like Human Computer Interaction, Human Behavior Understanding, Face Analysis and Video Surveillance. In last decades, many efforts have been conducted to develop accurate and reliable head or face detectors on standard RGB images, but only few solutions concern other types of images, such as depth maps. In this paper, we propose a novel method for head detection on depth images, based on a deep learning approach. In particular, the presented system overcomes the classic sliding-window approach, that is often the main computational bottleneck of many object detectors, through a Fully Convolutional Network. Two public datasets, namely Pandora and Watch-n-Patch, are exploited to train and test the proposed network. Experimental results confirm the effectiveness of the method, that is able to exceed all the state-of-art works based on depth images and to run with real time performance.

View PDFchevron_right
Person Head Detection in Multiple Scales Using Deep Convolutional Neural Networks
Muhammad Saqib

2018 International Joint Conference on Neural Networks (IJCNN), 2018

Person detection is an important problem in computer vision with many real-world applications. The detection of a person is still a challenging task due to variations in pose, occlusions and lighting conditions. The purpose of this study is to detect human heads in natural scenes acquired from a publicly available dataset of Hollywood movies. In this work, we have used state-of-the-art object detectors based on deep convolutional neural networks. These object detectors include region-based convolutional neural networks using region proposals for detections. Also, object detectors that detect objects in the single-shot by looking at the image only once for detections. We have used transfer learning for fine-tuning the network already trained on a massive amount of data. During the fine-tuning process, the models having high mean Average Precision (mAP) are used for evaluation of the test dataset. Experimental results show that Faster R-CNN [18] and SSD MultiBox [13] with VGG16 [21] perform better than YOLO [17] and also demonstrate significant improvements against several baseline approaches.

View PDFchevron_right
Face-from-Depth for Head Pose Estimation on Depth Images
Guido Borghi

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018

Depth cameras allow to set up reliable solutions for people monitoring and behavior understanding, especially when unstable or poor illumination conditions make unusable common RGB sensors. Therefore, we propose a complete framework for the estimation of the head and shoulder pose based on depth images only. A head detection and localization module is also included, in order to develop a complete end-to-end system. The core element of the framework is a Convolutional Neural Network, called POSEidon + , that receives as input three types of images and provides the 3D angles of the pose as output. Moreover, a Face-from-Depth component based on a Deterministic Conditional GAN model is able to hallucinate a face from the corresponding depth image. We empirically demonstrate that this positively impacts the system performances. We test the proposed framework on two public datasets, namely Biwi Kinect Head Pose and ICT-3DHP, and on Pandora, a new challenging dataset mainly inspired by the automotive setup. Experimental results show that our method overcomes several recent state-of-art works based on both intensity and depth input data, running in real-time at more than 30 frames per second.

View PDFchevron_right
FotonNet: A HW-Efficient Object Detection System Using 3D-Depth Segmentation and 2D-DNN Classifier
Gurjeet Singh

ArXiv, 2018

Object detection and classification is one of the most important computer vision problems. Ever since the introduction of deep learning \cite{krizhevsky2012imagenet}, we have witnessed a dramatic increase in the accuracy of this object detection problem. However, most of these improvements have occurred using conventional 2D image processing. Recently, low-cost 3D-image sensors, such as the Microsoft Kinect (Time-of-Flight) or the Apple FaceID (Structured-Light), can provide 3D-depth or point cloud data that can be added to a convolutional neural network, acting as an extra set of dimensions. In our proposed approach, we introduce a new 2D + 3D system that takes the 3D-data to determine the object region followed by any conventional 2D-DNN, such as AlexNet. In this method, our approach can easily dissociate the information collection from the Point Cloud and 2D-Image data and combine both operations later. Hence, our system can use any existing trained 2D network on a large image da...

View PDFchevron_right
Deep Head Pose Estimation from Depth Data for In-Car Automotive Applications
Guido Borghi

Lecture Notes in Computer Science, 2018

Recently, deep learning approaches have achieved promising results in various fields of computer vision. In this paper, we tackle the problem of head pose estimation through a Convolutional Neural Network (CNN). Differently from other proposals in the literature, the described system is able to work directly and based only on raw depth data. Moreover, the head pose estimation is solved as a regression problem and does not rely on visual facial features like facial landmarks. We tested our system on a well known public dataset, Biwi Kinect Head Pose, showing that our approach achieves state-of-art results and is able to meet real time performance requirements.

View PDFchevron_right

Related topics

  • Computer Science
  • Artificial Intelligence
  • Computer Vision
  • Face Detection
  • Depth Maps
  • Head Detection
  • RGB-HIS color model

    • Cited by

    Depth and Thermal Images in Face Detection - A Detailed Comparison Between Image Modalities
    Wiktor Mucha

    2022 the 5th International Conference on Machine Vision and Applications (ICMVA)

    Face detection is a well-known issue in image processing, and numerous studies are present in this field. A prominent part of the work is devoted to RGB images, leaving depth and thermal data with less interest. However, in some conditions like low-light areas where face detection is needed, non-RGB sensors might perform better. Also, mounting an additional RGB camera could be challenging or not possible, considering privacy concerns. In this work, current deep learning methodologies are employed to train depth and thermal detection models. The training is done using combined publicly available data that is processed by us for this purpose in order to create necessary annotations for a learning process. The resulting models are validated on a new trimodal dataset collected for this experiments purpose. It contains images captured with RGB, depth, and thermal sensors. Various scenes with single and multiple faces appearances can be found. The results show that non-RGB solutions can be applied in practice with highly robust accuracy and their efficiency is close to RGB detectors. However, their performance depends on the environment and that circumstances are described later in this article. CCS CONCEPTS • Computing methodologies → Object detection.

    View PDFchevron_right
    Manual Annotations on Depth Maps for Human Pose Estimation
    Guido Borghi

    Lecture Notes in Computer Science, 2019

    Few works tackle the Human Pose Estimation on depth maps. Moreover, these methods usually rely on automatically annotated datasets, and these annotations are often imprecise and unreliable, limiting the achievable accuracy using this data as ground truth. For this reason, in this paper we propose an annotation refinement tool of human poses, by means of body joints, and a novel set of fine joint annotations for the Watch-n-Patch dataset, which has been collected with the proposed tool. Furthermore, we present a fully-convolutional architecture that performs the body pose estimation directly on depth maps. The extensive evaluation shows that the proposed architecture outperforms the competitors in different training scenarios and is able to run in real-time.

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved