Academia.eduAcademia.edu

Figure 2 – uploaded by Karel Lebeda

See full PDFdownloadDownload figure
Table 1.2: Accents used within this work.

Table 1 2: Accents used within this work.

Related Figures (108)

Figure 1.1: Different possible object poses, shown on the sequence GYMNASTICS. From left to right: centre location, centre and size, axis-aligned bounding box, general rect- angular bounding box, pixel-level segmentation. tation propagation (see Figure 1.2 for an example of a shoppable video where user- Visual tracking has numerous applications such as video augmentation or anno-
Figure 1.1: Different possible object poses, shown on the sequence GYMNASTICS. From left to right: centre location, centre and size, axis-aligned bounding box, general rect- angular bounding box, pixel-level segmentation. tation propagation (see Figure 1.2 for an example of a shoppable video where user- Visual tracking has numerous applications such as video augmentation or anno-
Figure 1.2: Example of a tracker application: annotation propagation in a shoppable video.
Figure 1.2: Example of a tracker application: annotation propagation in a shoppable video.
Figure 1.3: Example of a tracker application: tracked trajectories used for dynamic scene analysis.
Figure 1.3: Example of a tracker application: tracked trajectories used for dynamic scene analysis.
Figure 1.4: Example of a tracker application: object stabilisation used for super- resolution. Top: selected frames of a video-sequence; bottom: detail of an input frame and of the output image.
Figure 1.4: Example of a tracker application: object stabilisation used for super- resolution. Top: selected frames of a video-sequence; bottom: detail of an input frame and of the output image.
Figure 1.5: Examples of a trackers internal models. Left: an edge quality field of the 2D LT-FLO tracker (Chapter 3); middle: a Gaussian-Process implicit model of the 3D TMAGIC tracker (Chapter 5); right: the same model made explicit and textured.
Figure 1.5: Examples of a trackers internal models. Left: an edge quality field of the 2D LT-FLO tracker (Chapter 3); middle: a Gaussian-Process implicit model of the 3D TMAGIC tracker (Chapter 5); right: the same model made explicit and textured.
Figure 1.7: Examples of common challenges within visual tracking (continued).
Figure 1.7: Examples of common challenges within visual tracking (continued).
Table 1.1: General notation used in this work. In this work, a general notation is used as tabulated in Table 1.1. Homogeneous shown, to which this thesis paves a way.
Table 1.1: General notation used in this work. In this work, a general notation is used as tabulated in Table 1.1. Homogeneous shown, to which this thesis paves a way.
Table 1.3: Feature types and typefaces used within this work.
Table 1.3: Feature types and typefaces used within this work.
Figure 2.1: Accuracy-robustness plot of the Visual Object Tracking challenge (VOT) 2013 results. The 95th percentiles of fitted normal distributions are shown, in the feature-based case with the LGT trackers (both vanilla and enhanced) excluded.
Figure 2.1: Accuracy-robustness plot of the Visual Object Tracking challenge (VOT) 2013 results. The 95th percentiles of fitted normal distributions are shown, in the feature-based case with the LGT trackers (both vanilla and enhanced) excluded.
Table 2.1: Comparison of state-of-the-art approaches for reconstruction of generic dy namic shapes. Inspired by [213].
Table 2.1: Comparison of state-of-the-art approaches for reconstruction of generic dy namic shapes. Inspired by [213].
Table 2.2: Comparison of the studied problems: SfM, SLAM and visual tracking. 2.4. Simultaneous Localisation and Mapping .4 Simultaneous Localisation and Mapping
Table 2.2: Comparison of the studied problems: SfM, SLAM and visual tracking. 2.4. Simultaneous Localisation and Mapping .4 Simultaneous Localisation and Mapping
Chapter 3. Long-Term 2D Tracking of Texture-less Objects highly variable due to changes in pose or scene illumination. While this is only true
Chapter 3. Long-Term 2D Tracking of Texture-less Objects highly variable due to changes in pose or scene illumination. While this is only true
Figure 3.2: Establishing line correspondences regardless of the aperture problem.
Figure 3.2: Establishing line correspondences regardless of the aperture problem.
Figure 3.3: Overview of the LT-FLOTrack algorithm. The second module maintains long-term relations. In cases of low confidence within
Figure 3.3: Overview of the LT-FLOTrack algorithm. The second module maintains long-term relations. In cases of low confidence within
on execution times, it should be as low as possible. To estimate a suitable number of Chapter 3. Long-Term 2D Tracking of Texture-less Objects correspondences for a given sequence, a data-driven method is proposed, which accounts for the object’s size and complexity; see Section 3.2.5 for details.
on execution times, it should be as low as possible. To estimate a suitable number of Chapter 3. Long-Term 2D Tracking of Texture-less Objects correspondences for a given sequence, a data-driven method is proposed, which accounts for the object’s size and complexity; see Section 3.2.5 for details.
Figure 3.4: Relationships and transitions between features. 3.2. Short-term Edge-based Tracking 3.2.3 Frame-to-Frame Transformation The Locally Optimised RANSAC (LO-RANSAC) [130] uses the correspondences
Figure 3.4: Relationships and transitions between features. 3.2. Short-term Edge-based Tracking 3.2.3 Frame-to-Frame Transformation The Locally Optimised RANSAC (LO-RANSAC) [130] uses the correspondences
Figure 3.5: Examples of an image and its edge quality field after several frames of tracking. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.5: Examples of an image and its edge quality field after several frames of tracking. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.6: Dependences of the number of correspondences and inliers on the number of generated points. The red line in the right image shows the standard deviation in bins of size 20. In this particular case (beginning of the DUDEK sequence), one can expect stable behaviour with about 350 correspondences, thus it is enough to generate 500 new points. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.6: Dependences of the number of correspondences and inliers on the number of generated points. The red line in the right image shows the standard deviation in bins of size 20. In this particular case (beginning of the DUDEK sequence), one can expect stable behaviour with about 350 correspondences, thus it is enough to generate 500 new points. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.8: Possible situations during correction. Thresholds 0; and @2 for the decisions are calculated from the object size. 3.4. Tracking Failure and Redetection
Figure 3.8: Possible situations during correction. Thresholds 0; and @2 for the decisions are calculated from the object size. 3.4. Tracking Failure and Redetection
Figure 3.10: Examples of probability distribution W (scale relative to the initial size). Top row: tracking a car, change only in the z-coordinate (visible steps in the upper left image indicate how narrow W is). Bottom row: tracking the PAGE sequence with large variation in rotation. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.10: Examples of probability distribution W (scale relative to the initial size). Top row: tracking a car, change only in the z-coordinate (visible steps in the upper left image indicate how narrow W is). Bottom row: tracking the PAGE sequence with large variation in rotation. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.11: Examples of probability distribution V, over x- and y-coordinates (for the same sequences as in Figure 3.10). Although the distributions on the left (estimated by KDE) are not unimodal, they are well modelled by the fitted Gaussian distribution (right column).
Figure 3.11: Examples of probability distribution V, over x- and y-coordinates (for the same sequences as in Figure 3.10). Although the distributions on the left (estimated by KDE) are not unimodal, they are well modelled by the fitted Gaussian distribution (right column).
Table 3.1: Results of LT-FLOTrack in the VOT2013 benchmark. The tabulated values are accuracy, robustness and speed in FPS. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Table 3.1: Results of LT-FLOTrack in the VOT2013 benchmark. The tabulated values are accuracy, robustness and speed in FPS. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Table 3.2: Overall results of the VOT Challenge 2013 [115]. The trackers are sorted according to combined performance. 3.0. Experimental Evaluation of LT-FLOTrack ~ sequences where a rigid object is tracked (CAP, JUICE). On the other hand, sequence:
Table 3.2: Overall results of the VOT Challenge 2013 [115]. The trackers are sorted according to combined performance. 3.0. Experimental Evaluation of LT-FLOTrack ~ sequences where a rigid object is tracked (CAP, JUICE). On the other hand, sequence:
Figure 3.12: Accuracy-robustness plot of the VOT Challenge 2013 results. For each tracker, the accuracy and robustness rankings were computed as a mean over all three experiments. Legend shown only for the best five trackers, see [115] for a full report. Figure 3.12: Accuracy-robustness plot of the VOT Challenge 2013 results. For each Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.12: Accuracy-robustness plot of the VOT Challenge 2013 results. For each tracker, the accuracy and robustness rankings were computed as a mean over all three experiments. Legend shown only for the best five trackers, see [115] for a full report. Figure 3.12: Accuracy-robustness plot of the VOT Challenge 2013 results. For each Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Table 3.3: Results of LT-FLOTrack in the VOT2014 benchmark. The tabulated values are accuracy, robustness and speed in FPS. 3.0. ~ Experimental Evaluation of LT-FLOTrack
Table 3.3: Results of LT-FLOTrack in the VOT2014 benchmark. The tabulated values are accuracy, robustness and speed in FPS. 3.0. ~ Experimental Evaluation of LT-FLOTrack
Figure 3.13: Accuracy-robustness plot of the VOT Challenge 2014 results. For each tracker, the accuracy and robustness rankings were computed as a mean over the two experiments. Legend shown only for the best five trackers, see [116] for a full report. Figure 3.13: Accuracy-robustness plot of the VOT Challenge 2014 results. For each Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.13: Accuracy-robustness plot of the VOT Challenge 2014 results. For each tracker, the accuracy and robustness rankings were computed as a mean over the two experiments. Legend shown only for the best five trackers, see [116] for a full report. Figure 3.13: Accuracy-robustness plot of the VOT Challenge 2014 results. For each Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Table 3.4: Overall results of the VOT Challenge 2014 [116]. The trackers are sorted according to overall performance. 3.5. Experimental Evaluation of LT-FLOTrack ~
Table 3.4: Overall results of the VOT Challenge 2014 [116]. The trackers are sorted according to overall performance. 3.5. Experimental Evaluation of LT-FLOTrack ~
Table 3.5: Results (AUC on the OPE test) on the VTB1.0 benchmark [206]. The trackers are sorted according to overal performance.
Table 3.5: Results (AUC on the OPE test) on the VTB1.0 benchmark [206]. The trackers are sorted according to overal performance.
Table 3.6: Experimental video sequences for short-term tracking. Sequences marked by “ are new and are available online with ground truth [118]. The largest challenge of the DUDEK sequence comes around the 210" frame, when evere problems, both in localisation and scale estimation.
Table 3.6: Experimental video sequences for short-term tracking. Sequences marked by “ are new and are available online with ground truth [118]. The largest challenge of the DUDEK sequence comes around the 210" frame, when evere problems, both in localisation and scale estimation.
Table 3.7: Speed comparison of different trackers (in FPS).
Table 3.7: Speed comparison of different trackers (in FPS).
Figure 3.14: Short-term tracking dataset (initialisation overlaid). From top to bottom: Doc [27], DuDEK [97], MuGN, PacEN and SpacrsutPN. 3.0. Experimental Evaluation of LT-FLOTrack =
Figure 3.14: Short-term tracking dataset (initialisation overlaid). From top to bottom: Doc [27], DuDEK [97], MuGN, PacEN and SpacrsutPN. 3.0. Experimental Evaluation of LT-FLOTrack =
Figure 3.15: Results of short-term tracking evaluation. From top to bottom: Doa. DUDEK, MuG, PAGE and SPACESHIP. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.15: Results of short-term tracking evaluation. From top to bottom: Doa. DUDEK, MuG, PAGE and SPACESHIP. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.16: Results of short-term tracking evaluation: the PAGE sequence and a de- tailed view of the most challenging part of the DUDEK sequence. 3.0. Experimental Evaluation of LT-FLOTrack =
Figure 3.16: Results of short-term tracking evaluation: the PAGE sequence and a de- tailed view of the most challenging part of the DUDEK sequence. 3.0. Experimental Evaluation of LT-FLOTrack =
Figure 3.17: Short-term tracking qualitative results. From top to bottom: Doc, DUDEK, MUG, PAGE and SPACESHIP. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.17: Short-term tracking qualitative results. From top to bottom: Doc, DUDEK, MUG, PAGE and SPACESHIP. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Table 3.8: Experimental video sequences for long-term tracking. Sequences marked by “ are new and are available online with ground truth [118]. Experimental Evaluation of LT-FLOTrack ~
Table 3.8: Experimental video sequences for long-term tracking. Sequences marked by “ are new and are available online with ground truth [118]. Experimental Evaluation of LT-FLOTrack ~
Figure 3.18: Long-term tracking dataset (initialisation overlaid). From top to bottom: CARCHASE [106], PANDA [106], VOLKSWAGEN [106], LiveRRuNN and NissanN. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.18: Long-term tracking dataset (initialisation overlaid). From top to bottom: CARCHASE [106], PANDA [106], VOLKSWAGEN [106], LiveRRuNN and NissanN. Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.19: Long-term tracking qualitative results. From top to bottom: LIVERRUN and NISSAN. 3.0. ~ Experimental Evaluation of LT-FLOTrack The proposed LT-FLO tracker is compared with TLD [106], the first explicitly long-
Figure 3.19: Long-term tracking qualitative results. From top to bottom: LIVERRUN and NISSAN. 3.0. ~ Experimental Evaluation of LT-FLOTrack The proposed LT-FLO tracker is compared with TLD [106], the first explicitly long-
Figure 3.20: Tracking without local corrections. Left: Doc, right: PAGE. 3.0. Experimental Evaluation of LT-FLOTrack = track the object throughout the sequence, with high errors reported very shortly after
Figure 3.20: Tracking without local corrections. Left: Doc, right: PAGE. 3.0. Experimental Evaluation of LT-FLOTrack = track the object throughout the sequence, with high errors reported very shortly after
Table 3.10: Effect of elements in Equation (3.18): short-term tracking (mean bounding box overlap tabulated) and long-term tracking (F-measure tabulated). Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Table 3.10: Effect of elements in Equation (3.18): short-term tracking (mean bounding box overlap tabulated) and long-term tracking (F-measure tabulated). Chapter 3. Long-Term 2D Tracking of Texture-less Objects
Figure 3.21: Sensitivity of the method to parameters: the learning threshold 0; (left: and the correction threshold 62 (right). 3.6. Closing Remarks on LT-FLOTrack
Figure 3.21: Sensitivity of the method to parameters: the learning threshold 0; (left: and the correction threshold 62 (right). 3.6. Closing Remarks on LT-FLOTrack
Figure 4.1: Top: selected frames of the BICYCLE sequence in the VOT Challenge (1, 140&173). Bottom: number of trackers from the challenge, which failed on particular frames. Notice the two challenging moments, a strong occlusion around frame 180 and an abrupt camera shake around frame 140. 4.1. Tracking within Scene Context
Figure 4.1: Top: selected frames of the BICYCLE sequence in the VOT Challenge (1, 140&173). Bottom: number of trackers from the challenge, which failed on particular frames. Notice the two challenging moments, a strong occlusion around frame 180 and an abrupt camera shake around frame 140. 4.1. Tracking within Scene Context
Figure 4.2: Selected frames from the DIVING sequence, challenging for many trackers, with overlaid “results” of the Zero-order Tracker [136].
Figure 4.2: Selected frames from the DIVING sequence, challenging for many trackers, with overlaid “results” of the Zero-order Tracker [136].
Figure 4.3: Comparison of discrete and differential entropy. Points were uniformly sampled from an interval (0;x) (x varies along the horizontal axis). The histogram bins for discrete entropy computation were fixed at integer positions. Notice how stable differential entropy is, even with sparsely distributed points (no interpolation used). Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.3: Comparison of discrete and differential entropy. Points were uniformly sampled from an interval (0;x) (x varies along the horizontal axis). The histogram bins for discrete entropy computation were fixed at integer positions. Notice how stable differential entropy is, even with sparsely distributed points (no interpolation used). Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.4: Illustration of information transfer between two processes X and Y. Chapter 4. Causality-based Motion Models in Visual Tracking TE is used here as the measure of the strength of a causal link. It is estimated on
Figure 4.4: Illustration of information transfer between two processes X and Y. Chapter 4. Causality-based Motion Models in Visual Tracking TE is used here as the measure of the strength of a causal link. It is estimated on
Figure 4.5: Left: Results of t-test for information transfer on synthetic data. Notice that very high statistical significance (p-value beyond any commonly used threshold) was reached even for random/invalid signal, due to non-independent samples. Right: Results of Welch’s t-test using surrogate data. 4.2. Measuring Causal Relationships: Transfer Entropy
Figure 4.5: Left: Results of t-test for information transfer on synthetic data. Notice that very high statistical significance (p-value beyond any commonly used threshold) was reached even for random/invalid signal, due to non-independent samples. Right: Results of Welch’s t-test using surrogate data. 4.2. Measuring Causal Relationships: Transfer Entropy
Figure 4.6: Dependence of TE on the time lag and the size of the time window. Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.6: Dependence of TE on the time lag and the size of the time window. Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.7: Dependence of TE on the time lag and the size of the time window. The column colours and the red line in (c) visualise the relative improvement R. The red stars denote all combinations of n and At with R > Op (1.e. where the condition from Eq. (4.6) is satisfied). Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.7: Dependence of TE on the time lag and the size of the time window. The column colours and the red line in (c) visualise the relative improvement R. The red stars denote all combinations of n and At with R > Op (1.e. where the condition from Eq. (4.6) is satisfied). Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.8: Window-based prediction function ¢, for the JUICE sequence. The training data are denoted by crosses and their associated colour, the background colour illus- trates prediction of y! (inter- and extrapolation from the training data), given y!, and t—At x'—At by the learned function ¢,,. In this case n = 1, however in general y‘’, and x’ can be high-dimensional. 4.3. Exploiting Causal Relationships: Causal Predictions =
Figure 4.8: Window-based prediction function ¢, for the JUICE sequence. The training data are denoted by crosses and their associated colour, the background colour illus- trates prediction of y! (inter- and extrapolation from the training data), given y!, and t—At x'—At by the learned function ¢,,. In this case n = 1, however in general y‘’, and x’ can be high-dimensional. 4.3. Exploiting Causal Relationships: Causal Predictions =
Figure 4.9: Predictions for the JUICE sequence. Black and red: the training data (crosses) and ground truth (in the extrapolated regions) of the X and Y signals, re- spectively; blue: mean and 95% confidence intervals (too narrow to be seen in (a)) of the prediction. The learned relationships ensure predictions of Y for frames 60-90 have higher accuracy and confidence than would be possible with simple extrapolation. Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.9: Predictions for the JUICE sequence. Black and red: the training data (crosses) and ground truth (in the extrapolated regions) of the X and Y signals, re- spectively; blue: mean and 95% confidence intervals (too narrow to be seen in (a)) of the prediction. The learned relationships ensure predictions of Y for frames 60-90 have higher accuracy and confidence than would be possible with simple extrapolation. Chapter 4. Causality-based Motion Models in Visual Tracking
Table 4.1: Causal detections on the VOT2013 dataset — detected durations and prop- erties. 4.5. Experimental Evaluation of Causality Detection ees
Table 4.1: Causal detections on the VOT2013 dataset — detected durations and prop- erties. 4.5. Experimental Evaluation of Causality Detection ees
Figure 4.12: Results on synthetic data (continued). A ground truth delay of 5 frames (At = 4 for n = 1) is used. The dotted lines in the second column denote approximate ground truth for Hx_,y (this approximation would be exact if the X and Y signals were normally distributed).
Figure 4.12: Results on synthetic data (continued). A ground truth delay of 5 frames (At = 4 for n = 1) is used. The dotted lines in the second column denote approximate ground truth for Hx_,y (this approximation would be exact if the X and Y signals were normally distributed).
Figure 4.13: Results on real data. The input signals c, p and p, are th and scene coordinate systems, respectively (only x-coordinate shown). e image position and the object position in the image TE and statistical significance using a range of lags are shown: (7) and (s) for image and scene coordinates, respectively. T] F from the input signal Y is denoted by the bold lines while the shuffled (Y) variants are denoted by the thin lines. For clarity, only x-signals in the case of the inputs, and only a small number of lags in the case of entropy and significance, are shown. Notice the different TE levels of the random signals, based on different input data. See the text for discussion.
Figure 4.13: Results on real data. The input signals c, p and p, are th and scene coordinate systems, respectively (only x-coordinate shown). e image position and the object position in the image TE and statistical significance using a range of lags are shown: (7) and (s) for image and scene coordinates, respectively. T] F from the input signal Y is denoted by the bold lines while the shuffled (Y) variants are denoted by the thin lines. For clarity, only x-signals in the case of the inputs, and only a small number of lags in the case of entropy and significance, are shown. Notice the different TE levels of the random signals, based on different input data. See the text for discussion.
Figure 4.14: Example of the time-based prediction on synthetic data. Notice how ¢; predicts p more accurately than ¢,. The narrower 95% confidence interval CI(¢;) also indicates a lower variance for the ¢; predictor. Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.14: Example of the time-based prediction on synthetic data. Notice how ¢; predicts p more accurately than ¢,. The narrower 95% confidence interval CI(¢;) also indicates a lower variance for the ¢; predictor. Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.15: Qualitative prediction results on the JUICE sequence, frame #400. Pre- dicted distributions shown with the ground truth position (in red) and ground truth inter-frame shift (blue) overlaid. See the text for discussion.
Figure 4.15: Qualitative prediction results on the JUICE sequence, frame #400. Pre- dicted distributions shown with the ground truth position (in red) and ground truth inter-frame shift (blue) overlaid. See the text for discussion.
Figure 4.16: Qualitative prediction results on the DIVING sequence, frame #200. Experimental Evaluation of Causal Predictions ~ equally important, based on the assumption that the object stays in an approximately
Figure 4.16: Qualitative prediction results on the DIVING sequence, frame #200. Experimental Evaluation of Causal Predictions ~ equally important, based on the assumption that the object stays in an approximately
Table 4.2: Quantitative results of the prediction on the VOT2013 dataset. The best and second best results are denoted by a old typeface and underlining respectively (separately for error expectation and mean probability density; multiple columns \ighlighted in cases of equal values).
Table 4.2: Quantitative results of the prediction on the VOT2013 dataset. The best and second best results are denoted by a old typeface and underlining respectively (separately for error expectation and mean probability density; multiple columns \ighlighted in cases of equal values).
Table 4.3: Tracking results on the VOT2013 benchmark. The best and second best results are highlighted separately for the FoT/Struck families of trackers and for accu- racy /robustness. Chapter 4. Causality-based Motion Models in Visual Tracking
Table 4.3: Tracking results on the VOT2013 benchmark. The best and second best results are highlighted separately for the FoT/Struck families of trackers and for accu- racy /robustness. Chapter 4. Causality-based Motion Models in Visual Tracking
Table 4.4: Tracking results on the VOT2013 benchmark with LT-FLOTrack.
Table 4.4: Tracking results on the VOT2013 benchmark with LT-FLOTrack.
Table 4.5: Tracking results on VTB1.1. Results in the last three columns were obtained using the VOT evaluation criteria, using the VTB criteria would improve the accuracy even further.
Table 4.5: Tracking results on VTB1.1. Results in the last three columns were obtained using the VOT evaluation criteria, using the VTB criteria would improve the accuracy even further.
Figure 4.17: Where is the mug? Causality-based prediction indicates its position even after the person turned away and completely occluded it. Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.17: Where is the mug? Causality-based prediction indicates its position even after the person turned away and completely occluded it. Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.18: Robust prediction during tracking failure on the ETH-CupP sequence of Grabner et al. [64].
Figure 4.18: Robust prediction during tracking failure on the ETH-CupP sequence of Grabner et al. [64].
Figure 4.19: Robust prediction during tracking failure on the CVMPMUG sequence. Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.19: Robust prediction during tracking failure on the CVMPMUG sequence. Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.20: Detail of the causal prediction, with individual windows contributions visualised. 4.8. Closing Remarks on Causal Relationships
Figure 4.20: Detail of the causal prediction, with individual windows contributions visualised. 4.8. Closing Remarks on Causal Relationships
Figure 4.21: Tracking and prediction results on the ETH-CupP sequence. Mean and confidence intervals (three standard deviations in dotted line) shown for human anno- tations. Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 4.21: Tracking and prediction results on the ETH-CupP sequence. Mean and confidence intervals (three standard deviations in dotted line) shown for human anno- tations. Chapter 4. Causality-based Motion Models in Visual Tracking
Figure 5.1: Out-of-plane rotations change the object appearance significantly, here is a complete change in just 50 frames. First row: original images. Second row: feature cloud and final model returned by the tracker. Notice the bottom and back side of the car, which have not been observed yet, so the point cloud does not reach there and the model is smoothly extrapolated. Chapter 5. Between Tracking and Structure from Motion
Figure 5.1: Out-of-plane rotations change the object appearance significantly, here is a complete change in just 50 frames. First row: original images. Second row: feature cloud and final model returned by the tracker. Notice the bottom and back side of the car, which have not been observed yet, so the point cloud does not reach there and the model is smoothly extrapolated. Chapter 5. Between Tracking and Structure from Motion
Figure 5.2: Overview of the TMAGIC tracker. Symbols on the right side of each step indicate inputs and outputs, as labelled for the case of 2D tracking. While the tracking oop is repeated in every frame, modelling only runs when necessary (according to Equation (5.4)). The numbers in upper left corners denote sections covering individual steps. Chapter 5. Between Tracking and Structure from Motion loop is repeated in every frame, modelling only runs when necessary (according to
Figure 5.2: Overview of the TMAGIC tracker. Symbols on the right side of each step indicate inputs and outputs, as labelled for the case of 2D tracking. While the tracking oop is repeated in every frame, modelling only runs when necessary (according to Equation (5.4)). The numbers in upper left corners denote sections covering individual steps. Chapter 5. Between Tracking and Structure from Motion loop is repeated in every frame, modelling only runs when necessary (according to
Figure 5.3: Life cycle of features used for 3D tracking. Chapter 5. Between Tracking and Structure from Motion The 2D point features xt € 4 are extracted (S1) using two techniques: Difference
Figure 5.3: Life cycle of features used for 3D tracking. Chapter 5. Between Tracking and Structure from Motion The 2D point features xt € 4 are extracted (S1) using two techniques: Difference
Figure 5.5: Examples of 3D feature clouds and GP-learned smooth models. Notice the unseen parts of the objects, which are without features and where the model is extrapolated (i.e. the rear side of the car and the “pole” on the left side of the cube). Figure 5.5: Examples of 3D feature clouds and GP-learned smooth models. Notice Chapter 5. Between Tracking and Structure from Motion
Figure 5.5: Examples of 3D feature clouds and GP-learned smooth models. Notice the unseen parts of the objects, which are without features and where the model is extrapolated (i.e. the rear side of the car and the “pole” on the left side of the cube). Figure 5.5: Examples of 3D feature clouds and GP-learned smooth models. Notice Chapter 5. Between Tracking and Structure from Motion
Figure 5.6: Star domain example in 2D. Left: Every region of the car shape can be reached from the centre without crossing the boundary, 7.e. its shape is a star domain. Right: Since there are regions unreachable by a straight line, it is not a star domain.
Figure 5.6: Star domain example in 2D. Left: Every region of the car shape can be reached from the centre without crossing the boundary, 7.e. its shape is a star domain. Right: Since there are regions unreachable by a straight line, it is not a star domain.
Figure 5.7: Iterative search for the shape centre. Black: training data. Coloured: shape centre (o) and sampled points (x), iterating from the centre of mass (blue) to convergence (green). Chapter 5. Between Tracking and Structure from Motion
Figure 5.7: Iterative search for the shape centre. Black: training data. Coloured: shape centre (o) and sampled points (x), iterating from the centre of mass (blue) to convergence (green). Chapter 5. Between Tracking and Structure from Motion
Figure 5.8: Iterative search for the shape centre with the RBF kernel; legend is the same as in Figure 5.7. Right: detail of regression near the shape centre.
Figure 5.8: Iterative search for the shape centre with the RBF kernel; legend is the same as in Figure 5.7. Right: detail of regression near the shape centre.
Figure 5.9: State of the proposed TMAGIC tracker after the first frame. Left: a bounding box, ¥! and L!. Right: 2, L, initial model M (the blue sphere visualises sampled points M) and C! (cnawenta), For the camera, the visualisation shows the projection centre C, principal direction and image plane. The upper left corner of the image plane is indicated by the dashed line.
Figure 5.9: State of the proposed TMAGIC tracker after the first frame. Left: a bounding box, ¥! and L!. Right: 2, L, initial model M (the blue sphere visualises sampled points M) and C! (cnawenta), For the camera, the visualisation shows the projection centre C, principal direction and image plane. The upper left corner of the image plane is indicated by the dashed line.
Figure 5.11: Selected frames from the CUBE1 sequence. Notice, how TMAGIC learns the new face of the cube. #194: unknown shape, the surface is smoothed over. #213: first features detected, shape roughly estimated. #239: more features identified, shape refined. #272: Final state, model in agreement with the object. Chapter 5. Between Tracking and Structure from Motion
Figure 5.11: Selected frames from the CUBE1 sequence. Notice, how TMAGIC learns the new face of the cube. #194: unknown shape, the surface is smoothed over. #213: first features detected, shape roughly estimated. #239: more features identified, shape refined. #272: Final state, model in agreement with the object. Chapter 5. Between Tracking and Structure from Motion
Figure 5.12: The 3D scene in t = 180 and 360. The camera and features are shown in the same way as in Figure 5.9, the camera trajectory (centres and principal directions for each previous frame) is shown in black. Ground-truth trajectory is in yellow. The details of the model can be seen in Figure 5.5.
Figure 5.12: The 3D scene in t = 180 and 360. The camera and features are shown in the same way as in Figure 5.9, the camera trajectory (centres and principal directions for each previous frame) is shown in black. Ground-truth trajectory is in yellow. The details of the model can be seen in Figure 5.5.
Figure 5.13: Deviation of the trajectory from a perfect circle (least-squares fitted). Errors are relative to the radius of the circle.
Figure 5.13: Deviation of the trajectory from a perfect circle (least-squares fitted). Errors are relative to the radius of the circle.
Figure 5.14: Selected frames from the test sequences (sequences from literature). From top: Doc, FISH, SYLVESTER and TWININGS. The initial bounding boxes are overlaid.
Figure 5.14: Selected frames from the test sequences (sequences from literature). From top: Doc, FISH, SYLVESTER and TWININGS. The initial bounding boxes are overlaid.
Figure 5.15: Selected frames from the test sequences (newly created sequences). From top: RALLY-LANCER, RALLY-VW, TOPGEARI1 and TOPGEAR2. The initial bounding boxes are overlaid. Chapter 5. Between Tracking and Structure from Motion
Figure 5.15: Selected frames from the test sequences (newly created sequences). From top: RALLY-LANCER, RALLY-VW, TOPGEARI1 and TOPGEAR2. The initial bounding boxes are overlaid. Chapter 5. Between Tracking and Structure from Motion
Table 5.1: Tracking results: mean localisation error (in pixels) and mean overlap (in percents). Bold numbers indicate the best result, underlined numbers the second best.
Table 5.1: Tracking results: mean localisation error (in pixels) and mean overlap (in percents). Bold numbers indicate the best result, underlined numbers the second best.
Figure 5.16: Qualitative results of the TMAGIC tracker. From top to bottom: RALLY- LANCER, RALLY-VW, TOPGEARI.
Figure 5.16: Qualitative results of the TMAGIC tracker. From top to bottom: RALLY- LANCER, RALLY-VW, TOPGEARI.
Figure 5.17: Visualisation of results of the quantitative performance analysis. From top and left: Doc, FISH, SYLVESTER, TWININGS, RALLY-LANCER, RALLY-VW, TOPGEARI and TOPGEAR2.
Figure 5.17: Visualisation of results of the quantitative performance analysis. From top and left: Doc, FISH, SYLVESTER, TWININGS, RALLY-LANCER, RALLY-VW, TOPGEARI and TOPGEAR2.
Figure 5.18: Results on the TORUS sequence: a failure case. Chapter 5. Between Tracking and Structure from Motion cause TMAGIC to fail immediately, as correct line features were still present, but it
Figure 5.18: Results on the TORUS sequence: a failure case. Chapter 5. Between Tracking and Structure from Motion cause TMAGIC to fail immediately, as correct line features were still present, but it
Chapter 6. Dense 3D Tracking of Non-Rigid Objects Figure 6.1: Example of input sequence and models output by the proposed method: frames #60, 70, 80 and 90 of the sequence CUBICGLOBE.
Chapter 6. Dense 3D Tracking of Non-Rigid Objects Figure 6.1: Example of input sequence and models output by the proposed method: frames #60, 70, 80 and 90 of the sequence CUBICGLOBE.
Figure 6.2: The HILLCLIMB sequence, a broadcast video divided into many sub- sequences by shot cuts. We provide an automatic modelling algorithm working through these sub-sequences.
Figure 6.2: The HILLCLIMB sequence, a broadcast video divided into many sub- sequences by shot cuts. We provide an automatic modelling algorithm working through these sub-sequences.
Figure 6.3: Overview of the proposed dense non-rigid tracking and reconstruction scheme. Chapter 6. Dense 3D Tracking of Non-Rigid Objects loop around the TMAGIC processing loop from Figure 5.2 (which is executed on each
Figure 6.3: Overview of the proposed dense non-rigid tracking and reconstruction scheme. Chapter 6. Dense 3D Tracking of Non-Rigid Objects loop around the TMAGIC processing loop from Figure 5.2 (which is executed on each
Figure 6.5: Sparse GP model (made explicit) and dense polygonal model. Chapter 6. Dense 3D Tracking of Non-Rigid Objects 6.2.2 Final model extraction
Figure 6.5: Sparse GP model (made explicit) and dense polygonal model. Chapter 6. Dense 3D Tracking of Non-Rigid Objects 6.2.2 Final model extraction
Figure 6.6: Resulting models on sequences from literature with varying resolution. From top to bottom: DoG1l (90 px [27]), SYLVESTER (50 px [166]) and RALLY-VW (410 px, Chapter 5). Experimental Evaluation of Dense Modelling ~
Figure 6.6: Resulting models on sequences from literature with varying resolution. From top to bottom: DoG1l (90 px [27]), SYLVESTER (50 px [166]) and RALLY-VW (410 px, Chapter 5). Experimental Evaluation of Dense Modelling ~
Figure 6.7: Results on the RALLY-VW sequence. Top and bottom rows show the top and side views of models from the same method (the right column is manually cropped for CMP results). Top to bottom: CMP SfM WebService [81] (raw video input); CMP Sf{M WebService [81] with added information; textured sparse GP model; full final model. Chapter 6. Dense 3D Tracking of Non-Rigid Objects
Figure 6.7: Results on the RALLY-VW sequence. Top and bottom rows show the top and side views of models from the same method (the right column is manually cropped for CMP results). Top to bottom: CMP SfM WebService [81] (raw video input); CMP Sf{M WebService [81] with added information; textured sparse GP model; full final model. Chapter 6. Dense 3D Tracking of Non-Rigid Objects
Figure 6.8: Resulting models on several sub-sequences from the HILLCLIMB sequence. Experimental Evaluation of Dense Modelling ~
Figure 6.8: Resulting models on several sub-sequences from the HILLCLIMB sequence. Experimental Evaluation of Dense Modelling ~
Figure 6.9: Shot initialisation after the first three cuts in the HILLCLIMB sequence. Left: the last frame of a previous shot with the model overlaid; middle: the first frame of a new shot; right: the model is registered with the new frame. Chapter 6. Dense 3D Tracking of Non-Rigid Objects
Figure 6.9: Shot initialisation after the first three cuts in the HILLCLIMB sequence. Left: the last frame of a previous shot with the model overlaid; middle: the first frame of a new shot; right: the model is registered with the new frame. Chapter 6. Dense 3D Tracking of Non-Rigid Objects
Figure 6.10: Progressive growth of the model after processing particular shots from HILLCLIMB. The first two models and the final model are shown.
Figure 6.10: Progressive growth of the model after processing particular shots from HILLCLIMB. The first two models and the final model are shown.
6.3. Experimental Evaluation of Dense Modelling es
6.3. Experimental Evaluation of Dense Modelling es
Figure 6.11: Distribution of errors on the CUBICGLOBE sequence. Table 6.1: Quantitative tracking and reconstruction results on the CUBICGLOBE se- quence. See the text for discussion.
Figure 6.11: Distribution of errors on the CUBICGLOBE sequence. Table 6.1: Quantitative tracking and reconstruction results on the CUBICGLOBE se- quence. See the text for discussion.
Table 6.2: Times of processing the first 180 frames of the CUBICGLOBE sequence. The last row does not sum up to 100% due to various overhead computations, visualisation, I/O wait, etc. Figure 6.12: Mixing coefficients a’ in the CUBICGLOBE sequence. The dotted signals show the GT.
Table 6.2: Times of processing the first 180 frames of the CUBICGLOBE sequence. The last row does not sum up to 100% due to various overhead computations, visualisation, I/O wait, etc. Figure 6.12: Mixing coefficients a’ in the CUBICGLOBE sequence. The dotted signals show the GT.
Figure 6.13: ] Example of modelling results on the 300VW:002 sequence. From top to bottom: original video frames; video frames overlaid with the instantaneous models; the instantaneous model in its estimated pose (textured and untextured); the instantaneous model facing orward with modified texture. 6.3. Experimental Evaluation of Dense Modelling ik
Figure 6.13: ] Example of modelling results on the 300VW:002 sequence. From top to bottom: original video frames; video frames overlaid with the instantaneous models; the instantaneous model in its estimated pose (textured and untextured); the instantaneous model facing orward with modified texture. 6.3. Experimental Evaluation of Dense Modelling ik
Figure 6.14: Example of modelling results on the 300VW:004 sequence. From top to bottom: original video frames; video frames overlaid with the instantaneous models; the instantaneous model in its estimated pose (textured and untextured); the instantaneous model facing forward. Chapter 6. Dense 3D Tracking of Non-Rigid Objects
Figure 6.14: Example of modelling results on the 300VW:004 sequence. From top to bottom: original video frames; video frames overlaid with the instantaneous models; the instantaneous model in its estimated pose (textured and untextured); the instantaneous model facing forward. Chapter 6. Dense 3D Tracking of Non-Rigid Objects
Figure 6.15: Basis shapes in the non-rigid model produced by the proposed algorithm. 6.3. Experimental Evaluation of Dense Modelling
Figure 6.15: Basis shapes in the non-rigid model produced by the proposed algorithm. 6.3. Experimental Evaluation of Dense Modelling
Figure 6.16: Reconstructed model overlaid over frames from the 300VW:002 sequence. The shape space visualises the weighted combination of the basis shapes. Chapter 6. Dense 3D Tracking of Non-Rigid Objects
Figure 6.16: Reconstructed model overlaid over frames from the 300VW:002 sequence. The shape space visualises the weighted combination of the basis shapes. Chapter 6. Dense 3D Tracking of Non-Rigid Objects
Figure 6.17: Comparison of BALM (left 2 columns) against the proposed technique (right 2 columns) on the 300VW:002 sequence. Results are shown using only the sparse supervision (top row), and using the sparse supervision with additional densely estimated trajectories (bottom row).
Figure 6.17: Comparison of BALM (left 2 columns) against the proposed technique (right 2 columns) on the 300VW:002 sequence. Results are shown using only the sparse supervision (top row), and using the sparse supervision with additional densely estimated trajectories (bottom row).
Figure 6.18: Landmark tracking error on the 300VW:002 sequence, when using rigid (i.e. vanilla TMAGIC) and non-rigid tracking and reconstruction. Left: error his- togram, right: landmark error from low (green) to high (red). Experimental Evaluation of Dense Modelling ~
Figure 6.18: Landmark tracking error on the 300VW:002 sequence, when using rigid (i.e. vanilla TMAGIC) and non-rigid tracking and reconstruction. Left: error his- togram, right: landmark error from low (green) to high (red). Experimental Evaluation of Dense Modelling ~
Figure 6.19: Example of modelling results on the FACE sequence. From top to bottom: original video frames; video frames overlaid with the instantaneous models; the in- stantaneous model in its estimated pose (textured and untextured); the instantaneous model facing forward. Chapter 6. Dense 3D Tracking of Non-Rigid Objects original video frames; video frames overlaid with the instantaneous models; the in-
Figure 6.19: Example of modelling results on the FACE sequence. From top to bottom: original video frames; video frames overlaid with the instantaneous models; the in- stantaneous model in its estimated pose (textured and untextured); the instantaneous model facing forward. Chapter 6. Dense 3D Tracking of Non-Rigid Objects original video frames; video frames overlaid with the instantaneous models; the in-
Figure 6.20: Example of modelling results on the T-SHIRT sequence. From top to bottom: original video frames; video frames overlaid with the instantaneous models; the instantaneous model in its estimated pose (textured and untextured); the instantaneous model facing forward. 6.4. Closing Remarks on Dense Tracking and Modelling
Figure 6.20: Example of modelling results on the T-SHIRT sequence. From top to bottom: original video frames; video frames overlaid with the instantaneous models; the instantaneous model in its estimated pose (textured and untextured); the instantaneous model facing forward. 6.4. Closing Remarks on Dense Tracking and Modelling
Figure 6.21: Details of the model obtained (directly) from the T-SHIRT sequence. Chapter 6. Dense 3D Tracking of Non-Rigid Objects
Figure 6.21: Details of the model obtained (directly) from the T-SHIRT sequence. Chapter 6. Dense 3D Tracking of Non-Rigid Objects
Figure 6.22: Basis shapes obtained from the FACE sequence.
Figure 6.22: Basis shapes obtained from the FACE sequence.
Related topics:
Computer ScienceComputer VisionObject Tracking (Computer Vision)Gaussian processesOptical FlowTransfer Entropy
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved