Multimodal workbench for automatic surveillance applications

Procedia Computer Science, 2015

Human-machine interaction mixed initiatives require a pragmatic coordination between different systems. Context understanding is established from the content, analysis, and guidance from querybased coordination between users and machines. Inspired by Level 5 Information Fusion 'user refinement', a live-video computing (LVC) structure is presented for user-based query access of a data-base management of information. Information access includes multimedia fusion of query-based text, images, and exploited tracks which can be utilized for context assessment, content-based information retrieval (CBIR), and situation awareness. In this paper, we explore new developments in dynamic data-driven application systems (DDDAS) of context analysis for user support. Using a common image processing data set, a system-level time savings is demonstrated using a query-based approach in a context, control, and semantic-aware information fusion design.

Human–Computer Interaction Series, 2013

In this short paper we will present our latest research on a new framework being developed for aiding novice designers of highly interactive, cooperative, multimodal systems to make expert decisions in choice of interaction modalities depending on the type of activity and its cooperative nature. Our research is conducted within the field of maritime surveillance at ATOL Laboratory and it is focused on the next generation distributed work support.

2010

ABSTRACT In future military missions, there will be many sensor assets collecting much important information about the environment. User control over surveillance assets is important to ensure that the specific data collected is appropriate for the current mission. Unfortunately, previous work has shown that individual users cannot effectively control more than about four assets, even if the assets have significant autonomy.

2004

ADVISOR is an automated visual surveillance system for metro stations which was developed as part of the project ADVISOR, involving 3 academic and 3 industrial project partners. The ADVISOR system aims at making public transport safer by automatically detecting at an early stage dangerous situations which may lead to accidents, violence or vandalism. In order to achieve this people are tracked across the station and their behaviours analysed. Additional measurements on crowd density and movement are also obtained. Warnings are generated and displayed to human operators for possible intervention. The article explores the main difficulties encountered during the design and implementation of ADVISOR and describes the ways in which they were solved. A prototype system has been built and extensively tested, proving the feasibility of automated visual surveillance systems. An analysis of test runs at a metro station in Barcelona and several individual experiments show that the system cope...

IEEE Signal Processing Magazine, 2000

2009

Video surveillance systems must support multiple streaming and prompt alert notification. We propose a two-tiered environment: a supervisor defines presentation layouts and model interface reactions to alerts; a surveillant watches synchronized videos, adapts layouts, and is notified with alerts.

IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017

Wide-area monitoring for a smart community can be challenging in systems engineering because of its large scale and heterogeneity at the sensor, algorithm, and visualization levels. A smart interface to visualize high-level information fused from a diversity of low-level surveillance data, and to facilitate rapid response of events, is critical for the design of the system. This paper presents an event-driven visualization mechanism fusing multimodal information for a large-scale intelligent video surveillance system. The mechanism proactively helps security personnel intuitively be aware of events through close cooperation among visualization, data fusion, and sensor tasking. The visualization not only displays 2-D, 3-D, and geographical information within a condensed form of interface but also automatically shows the only important video streams corresponding to spontaneous alerts and events by a decision process called display switching arbitration. The display switching arbitration decides the importance of cameras by score ranking that considers event urgency and semantic object features. This system has been successfully deployed in a campus to demonstrate its usability and efficiency for an installation with two camera clusters that include dozens of cameras, and with a lot of video analytics to detect alerts and events. A further simulation comparing the display switching arbitration with similar camera selection methods shows that our method improves the visualization by selecting better representative camera views and reducing redundant switchover among multiview videos.

Lecture Notes in Computer Science, 2006

Coopetitive' interaction strategy has been shown to give better results than similar strategies like 'only cooperation ', 'only competition' etc [7]. However, this has been studied only in the context of visual sensors and for handling non-simultaneous events. In this paper, we study this 'coopetitive' strategy from a multimedia surveillance system perspective, wherein the system needs to utilize multiple heterogeneous sensors and also handle multiple simultaneous events. Applying such an interaction strategy to multimedia surveillance systems is challenging because heterogeneous sensors have different capabilities for performing different sub-tasks as well as dissimilar response times. We adopt a merit-cum-availability based approach to allocate various sub-tasks to the competing sensors which eventually cooperate to achieve the specified system goal. Also, a 'coopetition' based strategy is adopted for effectively utilizing the information coming asynchronously from different data sources. Multiple simultaneous events (e.g. multiple intrusions) are handled by adopting a predictive strategy which estimates the exit time for each intruder and then uses this information for enhanced scheduling. The results obtained for two sets of surveillance experiments conducted with two active cameras and a motion sensor grid are promising.

Human–Computer Interaction, 2010

One possibility presented by novel communication technologies is the ability for remotely located experts to provide guidance to others who are performing difficult technical tasks in the real world, such as medical procedures or engine repair. In these scenarios, video views and other visual information seem likely to be useful in the ongoing negotiation of shared understanding, or common ground, but actual results with experimental systems have been mixed. One difficulty in designing these systems is achieving a balance between close-up ...

Multimedia Tools and Applications, 2013

Automated multimedia surveillance systems capture, process and analyze multimedia data coming from heterogeneous sensors. These systems are often designed to support (semi-) automatic decision making, such as generating an alarm in response to a surveillance event, as well as providing useful information to human decision makers to ensure public safety. Various tools and techniques from different fields such as Computer Vision, Pattern Recognition, and Multimedia Computing have contributed to the success of such systems. Although there has been significant progress in the field of multimedia surveillance research, we still face situations when the system is unable to detect critical events, wrongly identifies individuals or generates false alarms leading to undesired consequences. Hence, the goal of this special issue is to bring forward recent advancements in automated multimedia surveillance for improved public safety. More specifically, it reports the state-of-the-art techniques, methodologies and systems for automatic surveillance aiming to protect people and ensure their safety. An open call for papers attracted 16 submissions addressing different aspects of multimedia surveillance systems. After a thorough and highly competitive review process, the guest editorial committee recommended the acceptance of 11 top quality papers, though most submissions were of high quality. The accepted papers