Academia.eduAcademia.edu

Figure 3 – uploaded by A Ahmed

See full PDFdownloadDownload figure
Table 4 Timing analysis for the proposed F-2-F (global and local features) vs. trajectories For timing analysis against baselines, Table 4 shows video matching times for the revised F-2-F (local features on DC-image) against the main baseline F-2-F (global features on full image), the regenerated baseline F-2-F(global features on DC-image) and trajectories. It is noticeable that, the main baseline F-2-F(global features on full image) is the most costly, as it involves decompressing of full frames which is a lengthy process and makes the entire approach not suitable for speedy matching. The revised F-2-F (local features on DC-image) comes as the second highest (best accuracy), achieving 56 % time reduction over the main F-2-F baseline. Finally, trajectories and the regenerated baseline F-2-F(global features on DC-image) comes at the end, as the fastest techniques (with the lower accuracy), with only 0.2 and 0.5 seconds higher than the revised F-2-F. This extra time for the revised F-2-F (compared to the fastest), is due to the exhaustive comparison of SIFT keypoints among all possible frame pairs to fill the initial frame similarity matrix. However, the technique still works in real-time margin, and several optimizations could be done e.g. optimizing the code, reducing the number of matching frame pairs to fill the initial similarity matrix, replacing the dynamic programming algorithm by a faster one and improving the sigma adjustment process for a faster performance. — "1 +t: 1 xz ov +. ms aim ae ~ 7r 1. ag 1

Table 4 Timing analysis for the proposed F-2-F (global and local features) vs. trajectories For timing analysis against baselines, Table 4 shows video matching times for the revised F-2-F (local features on DC-image) against the main baseline F-2-F (global features on full image), the regenerated baseline F-2-F(global features on DC-image) and trajectories. It is noticeable that, the main baseline F-2-F(global features on full image) is the most costly, as it involves decompressing of full frames which is a lengthy process and makes the entire approach not suitable for speedy matching. The revised F-2-F (local features on DC-image) comes as the second highest (best accuracy), achieving 56 % time reduction over the main F-2-F baseline. Finally, trajectories and the regenerated baseline F-2-F(global features on DC-image) comes at the end, as the fastest techniques (with the lower accuracy), with only 0.2 and 0.5 seconds higher than the revised F-2-F. This extra time for the revised F-2-F (compared to the fastest), is due to the exhaustive comparison of SIFT keypoints among all possible frame pairs to fill the initial frame similarity matrix. However, the technique still works in real-time margin, and several optimizations could be done e.g. optimizing the code, reducing the number of matching frame pairs to fill the initial similarity matrix, replacing the dynamic programming algorithm by a faster one and improving the sigma adjustment process for a faster performance. — "1 +t: 1 xz ov +. ms aim ae ~ 7r 1. ag 1

Related Figures (16)

Fig. 1 Various types of video matching techniques
Fig. 1 Various types of video matching techniques
Fig. 2. Types of F-2-F matching, (a) unordered (b) ordered. In case of unequal video lengths, some frames will not be matched or might be matched to more than one frame depending on the matching algorithm [18]. The unordered matching is not common to be used as it discards the temporal order of video frames
Fig. 2. Types of F-2-F matching, (a) unordered (b) ordered. In case of unequal video lengths, some frames will not be matched or might be matched to more than one frame depending on the matching algorithm [18]. The unordered matching is not common to be used as it discards the temporal order of video frames
Fig. 3 SIFT/SURF local features extracted from DC-image (a) SIFT points extracted before adaptation (normal SIFT): 62 % of frames have less than 3 keypoints and after our SIFT adaptation (adaptive SIFT): 100 % of frames have enough keypoints for matching and (b) SURF points extracted from DC-image :62 % of frames have less than 3 keypoints. Results based on BBC Rushes [21]
Fig. 3 SIFT/SURF local features extracted from DC-image (a) SIFT points extracted before adaptation (normal SIFT): 62 % of frames have less than 3 keypoints and after our SIFT adaptation (adaptive SIFT): 100 % of frames have enough keypoints for matching and (b) SURF points extracted from DC-image :62 % of frames have less than 3 keypoints. Results based on BBC Rushes [21]
Fig. 4 Two real examples showing the underlying process of F-2-F technique in finding the optimal sequence of matching frames using dynamic programming approach, Horizontal axis is videol and vertical axis is video2. Due to the difference in videos lengths, some frames might not be matched. For example, frame 6 in video! in (a) and frames 2, 3, 5, 6, 10, 13 and 14 in videol in (b). The proposed algorithm carefully skips frames with minimum effect on finial matching cost In order to examine and evaluate the performance of our revised approach, we tested against various datasets; (1)BBC RUSHES (335 videos) [21], (2)UCF11 (1600 videos) [15] and (3)Mixed dataset of BBC RUSHES and UCF11 datasets (300 videos). The first is a stan- dard data set for video retrieval and contains a diverse set of challenging videos; mainly man-made moving objects (cars, tanks, planes and boats). The second is a standard dataset for action recognition and is widely used for retrieval purposes, as videos contain large variations in object appearance, pose, scale as well as camera movement. The third is a mix- ture of BBC RUSHES and UCF11. The performance is evaluated using precision-over-N
Fig. 4 Two real examples showing the underlying process of F-2-F technique in finding the optimal sequence of matching frames using dynamic programming approach, Horizontal axis is videol and vertical axis is video2. Due to the difference in videos lengths, some frames might not be matched. For example, frame 6 in video! in (a) and frames 2, 3, 5, 6, 10, 13 and 14 in videol in (b). The proposed algorithm carefully skips frames with minimum effect on finial matching cost In order to examine and evaluate the performance of our revised approach, we tested against various datasets; (1)BBC RUSHES (335 videos) [21], (2)UCF11 (1600 videos) [15] and (3)Mixed dataset of BBC RUSHES and UCF11 datasets (300 videos). The first is a stan- dard data set for video retrieval and contains a diverse set of challenging videos; mainly man-made moving objects (cars, tanks, planes and boats). The second is a standard dataset for action recognition and is widely used for retrieval purposes, as videos contain large variations in object appearance, pose, scale as well as camera movement. The third is a mix- ture of BBC RUSHES and UCF11. The performance is evaluated using precision-over-N
4.2 DC-image local features vs. global features
4.2 DC-image local features vs. global features
Fig. 5 a Matching precision for F-2-F based on global features considering the DC-image vs. full frame BBC Rushes, (b) F-2-F for DC-image and local features vs. global features for DC-image and full frame BBC Rushes, (c) F-2-F DC-image local features vs. global features UCF11 Following the results presented in previous Sections, 4.1 and 4.2, we conclude that the proposed F-2-F technique based on DC-image and local features is able to work in real-time manner. In addition, it achieved higher levels of accuracy compared to the same technique based on global features. This section empirically validate the proposed technique versus other baselines. We chose two baselines. The first is the trajectories [6] approach which encodes SIFT local features in the form of trajectories that capture the spatial and temporal eatures of a video shot. The second is the latest implementation of the F-2-F technique
Fig. 5 a Matching precision for F-2-F based on global features considering the DC-image vs. full frame BBC Rushes, (b) F-2-F for DC-image and local features vs. global features for DC-image and full frame BBC Rushes, (c) F-2-F DC-image local features vs. global features UCF11 Following the results presented in previous Sections, 4.1 and 4.2, we conclude that the proposed F-2-F technique based on DC-image and local features is able to work in real-time manner. In addition, it achieved higher levels of accuracy compared to the same technique based on global features. This section empirically validate the proposed technique versus other baselines. We chose two baselines. The first is the trajectories [6] approach which encodes SIFT local features in the form of trajectories that capture the spatial and temporal eatures of a video shot. The second is the latest implementation of the F-2-F technique
Fig. 6 The proposed F-2-F based on local features versus the regenerated F-2-F baseline, based on global features showing (a)2.3 % on average higher Py and 16.36 % more correct matches (from Fig. 7a), over the full BBC Rushes dataset, (b)20 % on average higher Py over the full mixed BBC Rushes and UCF11 dataset, using the mixed dataset gives more consolidation for the output results,(6)10.1 % on average higher Py over UCF11 dataset
Fig. 6 The proposed F-2-F based on local features versus the regenerated F-2-F baseline, based on global features showing (a)2.3 % on average higher Py and 16.36 % more correct matches (from Fig. 7a), over the full BBC Rushes dataset, (b)20 % on average higher Py over the full mixed BBC Rushes and UCF11 dataset, using the mixed dataset gives more consolidation for the output results,(6)10.1 % on average higher Py over UCF11 dataset
Fig. 7 Percent increase of correct matches (%) due to using F-2-F compared to: (a)the regenerated baseline and (b)trajectories baseline (values computed across top-10 ranks for each technique). Obviously F-2-F is able to retrieve more correct matches than mentioned baselines across all datasets the revised F-2-r, whe the original P-2-r applied on ruil size Trames. Starting with the second baseline (F-2-F with global features), it reported an average precision of 60 % based on five sample queries (only) out of 100 videos dataset. However, due to the difficulty of obtaining the dataset in order to obtain robust results based on a greater number of queries, we regenerated this baseline based on the BBC RUSHES, UCF11 and the mixed dataset of both, using the same color histogram global feature. For BBC RUSHES (Fig. 6a) although the regenerated baseline have higher Py across the 8!" — 10!” ranks, the revised F-2-F is better by 2.3 % on average overall ranks. Also, it is able to retrieve 16.36 % more correct matches, as depicted in Fig. 7a. For the mixed dataset the revised F- 2-F achieved 20 % on average higher Py compared to the regenerated baseline,as depicted in Fig. 6b. This corresponds to a 60.12 % increase in correct matches as showed in Fig. 7a.
Fig. 7 Percent increase of correct matches (%) due to using F-2-F compared to: (a)the regenerated baseline and (b)trajectories baseline (values computed across top-10 ranks for each technique). Obviously F-2-F is able to retrieve more correct matches than mentioned baselines across all datasets the revised F-2-r, whe the original P-2-r applied on ruil size Trames. Starting with the second baseline (F-2-F with global features), it reported an average precision of 60 % based on five sample queries (only) out of 100 videos dataset. However, due to the difficulty of obtaining the dataset in order to obtain robust results based on a greater number of queries, we regenerated this baseline based on the BBC RUSHES, UCF11 and the mixed dataset of both, using the same color histogram global feature. For BBC RUSHES (Fig. 6a) although the regenerated baseline have higher Py across the 8!" — 10!” ranks, the revised F-2-F is better by 2.3 % on average overall ranks. Also, it is able to retrieve 16.36 % more correct matches, as depicted in Fig. 7a. For the mixed dataset the revised F- 2-F achieved 20 % on average higher Py compared to the regenerated baseline,as depicted in Fig. 6b. This corresponds to a 60.12 % increase in correct matches as showed in Fig. 7a.
Table 3 Evaluation of F-2-F approach based on global features versus local features, for full image and DC-image respectively using unpaired t-test with confidence value of 95 %
Table 3 Evaluation of F-2-F approach based on global features versus local features, for full image and DC-image respectively using unpaired t-test with confidence value of 95 %
Fig. 8 the proposed F-2-F technique against trajectories baseline using (a)BBC RUSHES, showing 8.39 % on average higher Py, (b)Mixed BBC UCF11 dataset, showing 15.13 % on average higher preci- sion,(c)UCF11 dataset, showing 8.3 % an average higher Py across 4" ” 10" ranks, and an overall of 2.18 % increase in total correct matches (from Fig. 7b.)
Fig. 8 the proposed F-2-F technique against trajectories baseline using (a)BBC RUSHES, showing 8.39 % on average higher Py, (b)Mixed BBC UCF11 dataset, showing 15.13 % on average higher preci- sion,(c)UCF11 dataset, showing 8.3 % an average higher Py across 4" ” 10" ranks, and an overall of 2.18 % increase in total correct matches (from Fig. 7b.)
Fig. 9 Snapshot of video finder, using F2F based on local features (BBC_RUSHES) slower) is also available on http://dcapi.blogs.lincoln.ac.uk/F2F-demo/.
Fig. 9 Snapshot of video finder, using F2F based on local features (BBC_RUSHES) slower) is also available on http://dcapi.blogs.lincoln.ac.uk/F2F-demo/.
Fig. 10 Snapshot of video finder, using F2F based on local features (mixed BBC and UCF11)
Fig. 10 Snapshot of video finder, using F2F based on local features (mixed BBC and UCF11)
Saddam Bekhet is a PhD researcher at School of Computer Science in University of Lincoln, UK and an assistant lecturer in SouthValley University-Egypt. Received a MSc and BSc degrees in Computer Science at University of Assuit, Egypt in 2010 and 2005 respectively. Saddam’s current research focus on content based video retrieval, especially from the visual content perspective, video and scene analysis understanding.
Saddam Bekhet is a PhD researcher at School of Computer Science in University of Lincoln, UK and an assistant lecturer in SouthValley University-Egypt. Received a MSc and BSc degrees in Computer Science at University of Assuit, Egypt in 2010 and 2005 respectively. Saddam’s current research focus on content based video retrieval, especially from the visual content perspective, video and scene analysis understanding.
Amr Ahmed (BSc’93, MSc’98, PhD’04, MBCS’05, IEEE-CS’08) is a Senior Lecturer, and the Founder and the Leader of the DCAPI (Digital Contents Analysis, Production, and Interaction: http://dcapi.lincoln.ac. uk) research group at the School of Computer Science, University of Lincoln, UK. His research focuses on the analysis, understanding, and interpretation of digital contents, especially visual contents. Amr’s current research interests include Contents-Based Image/Video retrieval, video and scene understanding, semantic analysis, integration of knowledge and various modalities for scene understanding. Amr worked in the indus- try for several years, including Sharp Labs of Europe (SLE), Oxford (UK), as a Research Scientist, and other Engineering Consultants companies abroad. He also worked as a Research Fellow, at the University of Surrey, before joining the academic staff at the University of Lincoln in 2005. Dr. Ahmed is a Member of the British Computer Society (MBCS) and the IEEE Computer Society. He received his Bachelor’s degree in Electrical Engineering and M.Sc. degree (by research) in Computer and Systems Engineering, from Ain Shams University, Egypt, in 1993 and 1998 respectively, and his Ph.D. degree in Computer Graphics and Animation from the University of Surrey, U.K., in 2004.
Amr Ahmed (BSc’93, MSc’98, PhD’04, MBCS’05, IEEE-CS’08) is a Senior Lecturer, and the Founder and the Leader of the DCAPI (Digital Contents Analysis, Production, and Interaction: http://dcapi.lincoln.ac. uk) research group at the School of Computer Science, University of Lincoln, UK. His research focuses on the analysis, understanding, and interpretation of digital contents, especially visual contents. Amr’s current research interests include Contents-Based Image/Video retrieval, video and scene understanding, semantic analysis, integration of knowledge and various modalities for scene understanding. Amr worked in the indus- try for several years, including Sharp Labs of Europe (SLE), Oxford (UK), as a Research Scientist, and other Engineering Consultants companies abroad. He also worked as a Research Fellow, at the University of Surrey, before joining the academic staff at the University of Lincoln in 2005. Dr. Ahmed is a Member of the British Computer Society (MBCS) and the IEEE Computer Society. He received his Bachelor’s degree in Electrical Engineering and M.Sc. degree (by research) in Computer and Systems Engineering, from Ain Shams University, Egypt, in 1993 and 1998 respectively, and his Ph.D. degree in Computer Graphics and Animation from the University of Surrey, U.K., in 2004.
Amjad Altadmri received the PhD degree from the School of Computer Science at the University of Lincoln, UK in 2013. A Bachelor of Engineering in Computer Science from University of Damascus, Syria in 2004. Amjad’s current research focuses on Semantics of Visual contents, both from visual contents area and the link to the semantic textual one. His other research interests include Video Understanding, Ontology and Commonsense.
Amjad Altadmri received the PhD degree from the School of Computer Science at the University of Lincoln, UK in 2013. A Bachelor of Engineering in Computer Science from University of Damascus, Syria in 2004. Amjad’s current research focuses on Semantics of Visual contents, both from visual contents area and the link to the semantic textual one. His other research interests include Video Understanding, Ontology and Commonsense.
Andrew Hunter studied for his BSc Mathematics and Computing and PhD Computer Graphics at Bath University. He worked for several years in industry, in computer graphics and CAD/CAM software, before returning to -academia. Prof. Hunter held posts at Sunderland and -Durham Universities, before joining the University of Lincoln and becoming Head of Department of Computing in 2004, Dean of Research in 2007, and Dean for Science, Technology and Engineering in 2010. Professor Hunter has published over 80 academic papers, including more than 20 in international journals. He has also developed several free- ware and commercial artificial intelligence software packages. Professor Hunter’s research interests are in computer vision and artificial intelligence, and particularly in medical applications of vision and auto- mated surveillance. Major recent projects include: BRAINS, a TSB-sponsored project to develop embedded neurally-inspired smart sensor systems for surveillance based on FPGA technology; TOTALCARE, a TSB- sponsored project to develop an integrated automated in-home monitoring system for assistive care, including novel behaviour analysis; and retinal analysis, including detection and measurement of features of the reti- nal vasculature, and the application of level set methods and novel shape features in lesion detection and classification. He leads a small group of three Research Assistants and PhD students.
Andrew Hunter studied for his BSc Mathematics and Computing and PhD Computer Graphics at Bath University. He worked for several years in industry, in computer graphics and CAD/CAM software, before returning to -academia. Prof. Hunter held posts at Sunderland and -Durham Universities, before joining the University of Lincoln and becoming Head of Department of Computing in 2004, Dean of Research in 2007, and Dean for Science, Technology and Engineering in 2010. Professor Hunter has published over 80 academic papers, including more than 20 in international journals. He has also developed several free- ware and commercial artificial intelligence software packages. Professor Hunter’s research interests are in computer vision and artificial intelligence, and particularly in medical applications of vision and auto- mated surveillance. Major recent projects include: BRAINS, a TSB-sponsored project to develop embedded neurally-inspired smart sensor systems for surveillance based on FPGA technology; TOTALCARE, a TSB- sponsored project to develop an integrated automated in-home monitoring system for assistive care, including novel behaviour analysis; and retinal analysis, including detection and measurement of features of the reti- nal vasculature, and the application of level set methods and novel shape features in lesion detection and classification. He leads a small group of three Research Assistants and PhD students.
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved