Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Tools and Methods
Data Compression
Data Acquisition
Data Coding
Data Transmission
Data Visualization and Skeleton Tracking
Tests and Results
Conclusions
References
All Topics
Engineering
Instrumentation

INDOOR 3D VIDEO MONITORING USING MULTIPLE KINECT DEPTH-CAMERAS

Profile image of Mario Martinez-ZarzuelaMario Martinez-ZarzuelaProfile image of The International Journal of Multimedia (IJMA) - ERA IndexedThe International Journal of Multimedia (IJMA) - ERA Indexed
https://doi.org/10.5121/IJMA.2014.6105
visibility

…

description

16 pages

link

1 file

Abstract

This article describes the design and development of a system for remote indoor 3D monitoring using an undetermined number of Microsoft® Kinect sensors. In the proposed client-server system, the Kinect cameras can be connected to different computers, addressing this way the hardware limitation of one sensor per USB controller. The reason behind this limitation is the high bandwidth needed by the sensor, which becomes also an issue for the distributed system TCP/IP communications. Since traffic volume is too high, 3D data has to be compressed before it can be sent over the network. The solution consists in selfcoding the Kinect data into RGB images and then using a standard multimedia codec to compress color maps. Information from different sources is collected into a central client computer, where point clouds are transformed to reconstruct the scene in 3D. An algorithm is proposed to merge the skeletons detected locally by each Kinect conveniently, so that monitoring of people is robust to self and inter-user occlusions. Final skeletons are labeled and trajectories of every joint can be saved for event reconstruction or further analysis.

Related papers

AN APPROACH TO 3D MODEL APPLICATION WITH KINECT
Yusuf Uzun

Moving 3D modeling is widely used in many areas such as cinema, gaming, robot control and training. Kinect is a system that can detect human movements and send them to computers. It was developed by Microsoft to play games for the Xbox game console and is being used over time for applications in other areas. This work aims to create a 3D human model using kinect.

View PDFchevron_right
Real-Time Posture Reconstruction for Microsoft Kinect
Edmond Ho

2013

The recent advancement of motion recognition using Microsoft Kinect stimulates many new ideas in motion capture and virtual reality applications. Utilizing a pattern recognition algorithm, Kinect can determine the positions of different body parts from the user. However, due to the use of a single depth camera, recognition accuracy drops significantly when the parts are occluded. This hugely limits the

View PDFchevron_right
3D modeling of indoor environments using Kinect sensor
Ezzeddine Zagrouba

2013 IEEE Second International Conference on Image Information Processing (ICIIP-2013), 2013

3D scene modeling for indoor environments has stirred significant interest in the last few years. The obtained photo-realistic rendering of internal structures are being used in a huge variety of civilian and military applications such as training, simulation, patrimonies conservation, localization and mapping. Whereas, building such complicated maps poses significant challenges for both computer vision and robotic communities (low lighting and textureless structures, transparent and specular surfaces, registration and fusion problems, coverage of all details, real time constraint, etc.). Recently, the Microsoft Kinect sensors, originally developed as a gaming interface, have received a great deal of attention as being able to produce high quality depth maps in real time. However, we realized that these active sensors failed completely on transparent and specular surfaces due to many technical causes. As these objects should be involved into the 3D model, we have investigated methods to inspect them without any modification of the hardware. In particular, the Structure from Motion (SFM) passive technique can be efficiently integrated to the reconstruction process to improve the detection of these surfaces. In fact, we proposed to fill the holes in the depth map provided by the Infrared (IR) kinect sensor with new values passively retrieved by the SFM technique. This helps to acquire additional huge amount of depth information in a relative short time from two consecutive RGB frames. To conserve the real time aspect of our approach we propose to select key-RGB-images instead of using all the available frames. The experiments show a strong improvement in the indoor reconstruction as well as transparent object inspection.

View PDFchevron_right
A State of the Art Report on Kinect Sensor Setups in Computer Vision
Kai Berger

During the last three years after the launch of the Microsoft Kinect R in the end-consumer market we have become witnesses of a small revolution in computer vision research towards the use of a standardized consumer-grade RGBD sensor for scene content retrieval. Beside classical localization and motion capturing tasks the Kinect has successfully been employed for the reconstruction of opaque and transparent objects. This report gives a comprehensive overview over the main publications using the Microsoft Kinect out of its original context as a decision-forest based motion-capturing tool.

View PDFchevron_right
Indoor Human Tracking Application Using Multiple Depth-Cameras
Paulus Utama

2012

This research developed an application that could tracks and locates human's presence and position in indoor environment using multiple depth-cameras. Kinect as the most affordable device that equipped with depthcamera was used in this work. The application obtains stream data from Kinect and analyzes presence of human using skeletal tracking library on Kinect for Windows SDK v1. The final application also visualizes human location on 3D environment using Windows Presentation Foundation (WPF) 4.0. In order to visualize 3D object correctly, the application also took into account the coverage that may intersect when two Kinects were placed in adjacent position so that the final human location is combined. In the end, application was tested in 3 different scenarios and it's found that the average error in determining human location was 0.13589 meters.

View PDFchevron_right

References (26)

  1. Sage, K. & Young S. (1999) Security Applications of Computer Vision, Aerospace and Electronic Systems Magazine, IEEE 14(4):19-29.
  2. Boutaina, H., Rachid, O.H.T. & Mohammed, E.R.T. (2013) Tracking multiple people in real time based on their trajectory, in Proc. of Intelligent Systems: Theories and Applications (SITA), 8th International Conference on, pp.1-5, Rabat, Morocco.
  3. Strbac, M., Markoviu, M., Rigolin, L. & Popoviu, D.B. (2012) Kinect in Neurorehabilitation: Computer vision system for real time and object detection and instance estimation, in Proc. Neural Network Applications in Electrical Engineering (NEUREL), 11th Symposium on, pp. 127-132, Belgrade, Serbia.
  4. Martín Moreno, J., Ruiz Fernandez, D., Soriano Paya, A. & Berenguer Miralles, V. (2008) Monitoring 3D movements for the rehabilitation of joints in Physiotherapy, in Proc. Engineering in Medicine and Biology Society (EMBS), 30th Annual International Conference of the IEEE, pp. 4836- 4839, Vancouver, Canada.
  5. Ukita, N. & Matsuyama, T. (2002) Real-Time Cooperative Multi-Target Tracking by Communicating Active Vision Agents, in Proc. Pattern Recognition, 16th International Conference on, vol.2, pp.14- 19.
  6. R Bodor, R., Jackson, B. & Papanikolopoulos, N. (2003) Vision-Based Human Tracking and Activity Recognition, in Proc. of the 11th Mediterranean Conf. on Control and Automation, pp. 18-20.
  7. Poppe R. (2007). Vision-based human motion analysis: An overview, Computer Vision and Image Understanding 108:4-18.
  8. Weinland, D., Ronfard, R. & Boyer, E. (2011) A survey of vision-based methods for action representation, segmentation and recognition, Computer Vision and Image Understanding 115(2):224-241.
  9. Kinect for Windows (2013) Kinect for Windows. Retrieved from http://www.microsoft.com/en- us/kinectforwindows/develop/overview.aspx, last visited on July 2013.
  10. Burrus, N. (2013) RGBDemo. Retrieved from http://labs.manctl.com/rgbdemo/, last visited on July 2013.
  11. KinectTCP (2013) KinectTCP. Retrieved from https://sites.google.com/a/temple.edu/kinecttcp/, last visited on July 2013
  12. OpenNI (2013) OpenNI. Plataform to promote interoperability among devices, applications and Natural Interaction (NI) middleware. Retrieved from http://www.openni.org, last visited on July 2013.
  13. Prime Sense (2013) Prime Sense. Retrieved from http://www.primesense.com/, last visited June 2013.
  14. Martínez Rach, M.O., Piñol, P., López Granado, O. & Malumbres, M.P. (2012) Fast zerotree wavelet depth map encoder for very low bitrate, in Actas de las XXIII Jornadas de Paralelismo, Elche, Spain.
  15. Joon-Heup, K., Moon-Sang J. & Jong-Tae, P. (2001) An Efficient Naming Service for CORBA-based Network Management, in Integrated Network Management Proceedings, IEEE/IFIP International Symposium on, pp.765-778, Seattle, USA
  16. PCL (2013) Point Cloud Library. Retrieved from http://pointclouds.org/, last visited on July 2013.
  17. PCL Developers Blog (2013) PCL Developers Blog. Retrieved from http://pointclouds.org/blog/, last visited on July 2013.
  18. Parajuli, M., Tran, D.; Wanli, Ma;
  19. Sharma, D. (2012) Senior health monitoring using Kinect, in Proc. Communications and Electronics (ICCE), Fourth International Conference on, pp. 309-312, Hue, Vietnam.
  20. The WebM Project (2013) The WebM Project. Retrieved from http://www.webmproject.org, last visited on July 2013.
  21. Pece, F., Kautz, J. & Weyrich, T. (2011) Adapting Standard Video Codecs for Depth Streaming, in Proc. of the 17th Eurographics conference on Virtual Environments & Third Joint Virtual Reality (EGVE -JVRC), Sabine Coquillart, Anthony Steed, and Greg Welch (Eds.), pp. 59-66, Aire-la-Ville, Switzerland.
  22. García, J., Gardel, A., Bravo I., Lázaro, J.L., Martínez, M. & Rodríguez, D. (2013). Directional People Counter Based on Head Tracking, IEEE Transactions on Industrial Electronics 60(9): 3991- 4000.
  23. AUTHORS Mario Martínez-Zarzuela was born in Valladolid, Spain; in1979. He received the M.S. and Ph.D. degrees in telecommunication engineering from the University of Valladolid, Spain, in 2004 and 2009, respectively. Since 2005 he has been an assistant professor in the School of Telecommunication Engineering and a researcher in the Imaging & Telematics Group of the Department of Signal Theory, Communications and Telematics Engineering. His research interests include parallel processing on GPUs, computer vision, artificial intelligence, augmented and virtual reality, and natural human-computer interfaces.
  24. Miguel Pedraza Hueso was born in Salamanca, Spain, in 1990, He received his title in Telecommunication Engineering from the University of Valladolid, Spain, in 2013. Since 2011, he has collaborated with Imaging & Telematics Group of the Department of Signal Theory, Communications and Telematics Engineering. His research interests include applications with Kinect and augmented reality.
  25. Francisco Javier Díaz-Pernas was born in Burgos, Spain, in1962. He received the Ph.D. degree in industrial engineering from Valladolid University, Valladolid, Spain, in 1993. From 1988 to 1995, he joined the Department of System Engineering and Automatics, Valladolid University, Spain, where he has worked in artificial vision systems for industry applications as quality control for manufacturing. Since1996, he has been a professor in the School of Telecommunication Engineering and a Senior Researcher in Imaging & Telematics Group of the Department of Signal Theory, Communications, and Telematics Engineering. His main research interests are applications on the Web, intelligent transportation system, and neural networks for artificial vision.
  26. David González-Ortega was born in Burgos, Spain, in 1972. He received his M.S. and Ph.D. degrees in telecommunication engineering from the University of Valladolid, Spain, in 2002 and 2009, respectively. Since 2003 he has been a researcher in the Imaging and Telematics Group of the Department of Signal Theory, Communications and Telematics Engineering. Since 2005, he has been an assistant professor in the School of Telecommunication Engineering, University of Valladolid. His research interests include computer vision, image analysis, pattern recognition, neural networks, and real-time applications.

Related papers

DISTRIBUTED SYSTEM FOR 3D REMOTE MONITORING USING KINECT DEPTH CAMERAS
Computer Science & Information Technology (CS & IT) Computer Science Conference Proceedings (CSCP)

This article describes the design and development ofa system for remote indoor 3D monitoring using an undetermined number of Microsoft® Kinect sensors. In the proposed client-server system, the Kinect cameras can be connected to different computers, addressing this way the hardware limitation of one sensor per USB controller. The reason behind this limitation is the high bandwidth needed by the sensor, which becomes also an issue for the distributed system TCP/IP communications. Since traffic volume is too high, 3D data has to be compressed before it can be sent over the network. The solution consists in self-coding the Kinect data into RGB images and then using a standard multimedia codec to compress color maps. Information from different sources is collected into a central client computer, where point clouds are transformed to reconstruct the scene in 3D. An algorithm is proposed to conveniently merge the skeletons detected locally by each Kinect, so that monitoring of people is robust to self and inter-user occlusions. Final skeletons are labeled and trajectories of every joint can be saved for event reconstruction or further analysis.

View PDFchevron_right
3D with Kinect
Jan Smisek, Tomas Pajdla

2011

We analyze Kinect as a 3D measuring device, experimentally investigate depth measurement resolution and error properties and make a quantitative comparison of Kinect accuracy with stereo reconstruction from SLR cameras and a 3D-TOF camera. We propose Kinect geometrical model and its calibration procedure providing an accurate calibration of Kinect 3D measurement and Kinect cameras. We demonstrate the functionality of Kinect calibration by integrating it into an SfM pipeline where 3D measurements from a moving Kinect are transformed into a common coordinate system by computing relative poses from matches in color camera.

View PDFchevron_right
Dynamic scene reconstruction using asynchronous multiple Kinects
Ikuhisa Mitsugami

2012

This paper proposes a novel method to reconstruct dynamic scenes by integrating depth data obtained by multiple Kinects, which cannot be synchronized to one another. In this method, the multiple Kinects located so as to cover the whole surface are firstly calibrated so that their depth data are mapped into the world coordinate system. The synchronous depth data for each Kinect is then generated by interpolation of temporally neighboring captured data. Experimental results of marching person reconstruction show the effectiveness of our method.

View PDFchevron_right
Context-Aware 3D Gesture Interaction Based on Multiple Kinects
Julien Tscherrig

AMBIENT 2011, The …, 2011

This paper presents a novel context-aware system for deictic gestures interaction with smart environments. The system tracks multiple users; moreover, it recognizes inhabitants' postures and gestures in real-time. This information, enriched with smart objects coordinates, is reconstructed in a 3D model to allow the recognition process. Finally the system executes the programmed tasks to support the users' activity. Two Microsoft Kinect depth cameras have been used to acquire the data and a framework for the communication with the smart objects has been adopted. A first prototype has been developed and an evaluation test with 13 users has been conducted in order to assess the usability of the system. Results show that this interaction experience has been really appreciated by the users.

View PDFchevron_right
3D Reconstruction Technique with Kinect and Point Cloud Computing
Science Park Research Organization & Counselling

The background of this study using a variety of methods lies in providing a new interaction between them as the basis of virtual space with objects in the physical world. In this respect, the purpose is to cover in detail a scene for the reconstruction of 3D real-world objects. This involves the creation of virtual spaces using a 3D standard camera, the object of which is to display a 3D model. In order to achieve this target, during data acquisition, examination of the available methods and integration of multiple scan-related research were undertaken. As a result, in summary, this study involves the discussion of real-world objects as solid on the planar surfaces, which have been active in more than one depth-scanned point clouds (PCL), applying various filters to the background of the object, and then combining these scans.

View PDFchevron_right

Related topics

  • 3D Monitoring
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved