A Human Body Model for Articulated 3D Pose Tracking

Handbook of Ambient Intelligence and …, 2010

Visual tracking of unknown objects in unconstrained video-sequences is extremely challenging due to a number of unsolved issues. This thesis explores several of these and examines possible approaches to tackle them. The unconstrained nature of real-world input sequences creates huge variation in the appearance of the target object due to changes in pose and lighting. Additionally, the object can be occluded by either parts of itself, other elements of the scene, or the frame boundaries. Observations may also be corrupted due to low resolution, motion blur, large frame-to-frame displacement, or incorrect exposure or focus of the camera. Finally, some objects are inherently difficult to track due to their (low) texture, specular/transparent nature, non-rigid deformations, etc. Conventional trackers depend heavily on the texture of the target. This causes issues with transparent or untextured objects. Edge points can be used in cases where standard feature points are scarce; these however suffer from the aperture problem. To address this, the first contribution of this thesis explores the idea of virtual corners, using pairs of non-adjacent line correspondences, tangent to edges in the image. Furthermore, the chapter investigates the possibility of long-term tracking, introducing a re-detection scheme to handle occlusions while limiting drift of the object model. The outcome of this research is an edge-based tracker, able to track in scenarios including untextured objects, full occlusions and significant length. The tracker, besides reporting excellent results in standard benchmarks, is demonstrated to successfully track the longest sequence published to date. Some of the issues in visual tracking are caused by suboptimal utilisation of the image information. The object of interest can easily occupy as few as ten or even one percent of the video frame area. This causes difficulties in challenging scenarios such as sudden camera shakes or full occlusions. To improve tracking in such cases, the next major contribution of this thesis explores relationships within the context of visual tracking, with a focus on causality. These include causal links between the tracked object and other elements of the scene such as the camera motion or other objects. Properties of such relationships are identified in a framework based on information theory. The resulting technique can be employed as a causality-based motion model to improve the results of virtually any tracker. Significant effort has previously been devoted to rapid learning of object properties on the fly. However, state-of-the-art approaches still often fail in cases such as rapid out-of-plane rotations, when the appearance changes suddenly. One of the major contributions of this thesis is a radical rethinking of the traditional wisdom of modelling 3D motion as appearance change. Instead, 3D motion is modelled as 3D motion. This intuitive but previously unexplored approach provides new possibilities in visual tracking research. Firstly, 3D tracking is more general, as large out-of-plane motion is often fatal for 2D trackers, but helps 3D trackers to build better models. Secondly, the tracker's internal model of the object can be used in many different applications and it could even become the main motivation, with tracking supporting reconstruction rather than vice versa. This effectively bridges the gap between visual tracking and Structure from Motion. The proposed method is capable of successfully tracking sequences with extreme out-of-plane rotation, which poses a considerable challenge to 2D trackers. This is done by creating realistic 3D models of the targets, which then aid in tracking. In the majority of the thesis, the assumption is made that the target's 3D shape is rigid. This is, however, a relatively strong limitation. In the final chapter, tracking and dense modelling of non-rigid targets is explored, demonstrating results in even more generic (and therefore challenging) scenarios. This final advancement truly generalises the tracking problem with support for long-term tracking of low texture and non-rigid objects in sequences with camera shake, shot cuts and significant rotation. Taken together, these contributions address some of the major sources of failure in visual tracking. The presented research advances the field of visual tracking, facilitating tracking in scenarios which were previously infeasible. Excellent results are demonstrated in these challenging scenarios. Finally, this thesis demonstrates that 3D reconstruction and visual tracking can be used together to tackle difficult tasks.

One of the hard problems that combines research in Computer Graphics and Computer Vision is tracking human limbs in real time without using any markers. This problem is known to be notoriously difficult and ill posed. In this paper we present a technique to track human limbs at near interactive rates without markers. This work builds on, modifies, and adds to the state-of-theart tracking techniques developed by previous researchers. We use a particle filtering tracking algorithm that uses 3d data derived from a modified visual hull algorithm. The particle filtering algorithm runs on a GPU and hence we are able to track human limbs at 4Hz with reasonable accuracy. This method has the potential to be applied to tracking the whole human body without markers.

2014 International Conference on Electronics and Communication Systems (ICECS), 2014

Human tracking is a comprehensive framework for tracking coarse human model performed from sequences of synchronized monocular grayscale images in single or multiple camera system coordinates. It is nothing but segmenting an interested human from video scene and keep track if it continuously. It demonstrates the feasibility of an end to end person tracking system where initially it start background subtraction, then detection of the interested human and tracking of that human form one frame to another continuously. For detection of the interested human PCA algorithm is used. Finally Kalman filter is introduced into tracking the people. Our system have demonstrated that as compaired with other methods it reduces detection time comparitively and improves human detection and tracking accuracy.

2004

Automated 3-D tracking of the human body is a necessary prerequisite for interactive entertainment applications, video security systems, computer animation, bio-mechanical analysis and humancomputer interaction (e.g., gesture recognition). Currently, technologies use artificial markers and a feature tracking methodology to recover the target poses. In addition, most tracking systems alter the working environment and/or include multiple camera viewpoints in order to solve issues of occlusion and depth ambiguities. However, the constraint of artificial markers and the assumption of multiple viewpoint availability significantly reduce the range of operating environments for a tracking system, particularly in the analysis of archival video footage or in security monitoring applications. A single-view marker-less technique for both off-line and real-time on-line 3D tracking is presented and shown to be more than adequate for various 3D human body tracking applications. Key-Words: trackin...

2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06), 2000

We propose a novel method for tracking an articulated model in a 3D-point cloud. The tracking problem is formulated as the registration of two point sets, one of them parameterised by the model's state vector and the other acquired from a 3D-sensor system. Finding the correct parameter vector is posed as a linear estimation problem, which is solved by means of a scaled unscented Kalman filter. Our method draws on concepts from the widely used iterative closest point registration algorithm (ICP), basing the measurement model on point correspondences established between the synthesised model point cloud and the measured 3D-data. We apply the algorithm to kinematically track a model of the human upper body on a point cloud obtained through stereo image processing from one or more stereo cameras. We determine torso position and orientation as well as joint angles of shoulders and elbows. The algorithm has been successfully tested on thousands of frames of real image data. Challenging sequences of several minutes length where tracked correctly. Complete processing time remains below one second per frame.

Computer Vision – ECCV 2012, 2012

Tracking human pose in real-time is a difficult problem with many interesting applications. Existing solutions suffer from a variety of problems, especially when confronted with unusual human poses. In this paper, we derive an algorithm for tracking human pose in real-time from depth sequences based on MAP inference in a probabilistic temporal model. The key idea is to extend the iterative closest points (ICP) objective by modeling the constraint that the observed subject cannot enter free space, the area of space in front of the true range measurements. Our primary contribution is an extension to the articulated ICP algorithm that can efficiently enforce this constraint. The resulting filter runs at 125 frames per second using a single desktop CPU core. We provide extensive experimental results on challenging real-world data, which show that the algorithm outperforms the previous state-of-the-art trackers both in computational efficiency and accuracy.

Robotics and Automation, …, 2006

This paper proposes a tracking system called VooDoo for 3d tracking of human body movements based on a 3d body model and the Iterative Closest Point (ICP) algorithm. The proposed approach is able to incorporate raw data from different input sensors, as well as results from feature trackers in 2d or 3d. All input data is processed within the same model fitting step by modeling all input measurements in 3d model space. The system has been implemented and runs in realtime at appr. 10-14 Hz. Experiments with complex human movements exhibit the characteristics and advantages of the proposed approach.

The seminar and project Computer Vision: Object and People Tracking are continuative courses based on and applying the knowledge taught in the lectures 3D Computer Vision (INF-73-51-V-7) and Computer Vision: Object and People Tracking . The goal of the project is to research, design, implement and evaluate algorithms and methods for tackling computer vision problems. The seminar is more theoretical. Its educational objective is to train the ability to become acquainted with a specific research topic, review scientific articles and give a comprehensive presentation supported by media.

2006 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, 2006

In this paper, we present a new approach for fusion of different measurements and sensors for 3D model based tracking. The underlying model of the tracked body is defined geometrically with generalized cylinders, which can hierarchically be connected by different kinds of joints. This results in an articulated body model with constrained kinematic degrees of freedom. The fusion approach incorporates this model knowledge together with the measurements, and tracks the target body iteratively with an extended Iterative Closest Point approach. The resulting tracking system named VooDoo is used to track humans in a Human-Robot Interaction (HRI) context. We only rely on sensors on board the robot, i.e. a color camera, a 3d time-of-flight camera and a laser range finder. The system runs in realtime (∼ 20Hz) and is able to robustly track a human in the vicinity of the robot. The pose and trajectory of the human interaction partner can then be used for haptic interaction like handovers , and for activity and gesture recognition.