An expert system for surveillance picture understanding

Optics and Photonics for Counterterrorism, Crime Fighting, and Defence XII, 2016

The bottleneck in situation awareness is no longer in the sensing domain but rather in the data interpretation domain, since the number of sensors is rapidly increasing and it is not affordable to increase human data-analysis capacity at the same rate. Automatic image analysis can assist a human analyst by alerting when an event of interest occurs. However, common state-of-the-art image recognition systems learn representations in high-dimensional feature spaces, which makes them less suitable to generate a user-comprehensive message. Such data-driven approaches rely on large amounts of training data, which is often not available for quite rare but high-impact incidents in the security domain. The key contribution of this paper is that we present a novel real-time system for image understanding based on generic instantaneous low-level processing components (symbols) and flexible user-definable and user-understandable combinations of these components (sentences) at a higher level for the recognition of specific relevant events in the security domain. We show that the detection of an event of interest can be enhanced by utilizing recognition of multiple short-term preparatory actions.

Behavior Research Methods, 2006

Video security is becoming more and more vital today due to rapid development of hardware equipments as the number of installed cameras can confirm. This paper presents the system architecture of IVSS, an Intelligent Video Surveillance System based on IPcameras and deployed in an academic environment. In fact, Video surveillance is increasingly found in academic institutions. It is used to oversee the safety of teachers and students, as well as to protect assets from vandalism and theft. In this work the surveillance system is deployed in our university environment, it is based on a set of digital IP video cameras linked by the university IP network infrastructure. Our system architecture is based on efficient moving object detecting and tracking algorithm and a robust statistical activity recognition framework based on SVM which is used for modeling activities. The experimental results on real-time video streams show the feasibility of our system and its effectiveness in human activity recognition.

Emerging Research on Networked Multimedia Communication Systems, 2016

Smart surveillance systems must be able to detect and track moving objects, classify these objects and detect the activities. Automatic object detection is currently being embedded in smart surveillance systems. To achieve a higher level of semantic scene understanding, the objects and their actions have to be interpreted in the given context. One of the challenging aspects of automatically understanding a scene, which is still a subject of active research, is the presence of contextual information. In many cases, contextual information is required in order to correctly interpret the actions. This chapter explores the extraction of contextual information in several aspects such as spatial, motion, depth and co-occurrence, depending on applications. In this chapter, it is shown that using contextual information enables the automated analysis of complicated scenarios that was previously not possible using conventional object classification techniques.

IEEE Transactions on Consumer Electronics, 2000

With the continuous improvements in videoanalysis techniques, automatic low-cost video surveillance gradually emerges for consumer applications. Video surveillance can contribute to the safety of people in the home and ease control of home-entrance and equipmentusage functions. In this paper, we study a flexible framework for semantic analysis of human behavior from a monocular surveillance video, captured by a consumer camera. Successful trajectory estimation and human-body modeling facilitate the semantic analysis of human activities and events in video sequences. An additional contribution is the introduction of a 3-D reconstruction scheme for scene understanding, so that the actions of persons can be analyzed from different views. The total framework consists of four processing levels: (1) a preprocessing level including background modeling and multiple-person detection, (2) an object-based level performing trajectory estimation and posture classification, (3) an event-based level for semantic analysis, and (4) a visualization level including camera calibration and 3-D scene reconstruction. Our proposed framework was evaluated and has shown its good quality (86% accuracy of posture classification and 90% for events) and effectiveness, as it achieves a near real-time performance (6-8 frames/second). 1

With increased technology and population, surveillance is becoming key area in research. The best utilization of technology for the surveillance is the focus area today. The focus of the project would be to automate the detection of actions performed in front of camera. Monitoring the events on camera manually will not be possible. Even if the event had happened in the past, searching the same event in the video manually would cost a lot time. There is scope for automation which should detect those events and provide detailed textual result which can be processed easily. This project also considers effective way to store the video surveillance data consuming as small memory as possible along with smart index information. Video surveillance is an important security asset to control theft, traffic monitoring, banks, department stores, highways, crowded public places and borders.

2012

This paper presents a survey of Intelligent Visual Surveillance Systems (IVSS) with the aim to identify the needs and requirements of future large-scale (IVSS). Pros and cons of existing IVSS are discussed from different perspectives including: motion segmentation, region of interest (ROI) characterization, region of interest tracking, event recognition, and scenario profiling. Most IVSS approaches have limited fusion focus, superficial, and lack sufficient cohesiveness and completeness. By introducing sensor data and information fusion techniques, on the other hand, one can significantly improve the critical decision making at different levels of information abstraction and hence reducing decision uncertainty and improving ROI profiling in context-sensitive environments. In this paper, we have systematically addressed technological needs of different approaches supporting IVSS and identified a future direction to further enhance the current state-of-the-art of IVSS for homeland sec...

IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2004

Visual surveillance in dynamic scenes, especially for humans and vehicles, is currently one of the most active research topics in computer vision. It has a wide spectrum of promising applications, including access control in special areas, human identification at a distance, crowd flux statistics and congestion analysis, detection of anomalous behaviors, and interactive surveillance using multiple cameras, etc. In general, the processing framework of visual surveillance in dynamic scenes includes the following stages: modeling of environments, detection of motion, classification of moving objects, tracking, understanding and description of behaviors, human identification, and fusion of data from multiple cameras. We review recent developments and general strategies of all these stages. Finally, we analyze possible research directions, e.g., occlusion handling, a combination of twoand three-dimensional tracking, a combination of motion analysis and biometrics, anomaly detection and behavior prediction, content-based retrieval of surveillance videos, behavior understanding and natural language description, fusion of information from multiple sensors, and remote surveillance.

2007

A Multimedia Surveillance System (MSS) is considered for automatically retrieving semantic content from complex outdoor scenes, involving both human behavior and traffic domains. To characterize the dynamic information attached to detected objects, we consider a deterministic modeling of spatio-temporal features based on abstraction processes towards fuzzy logic formalism. A situational analysis over conceptualized information will not only allow us to describe human actions within a scene, but also to suggest possible interpretations of the behaviors perceived, such as situations involving thefts or dangers of running over. Towards this end, the different levels of semantic knowledge implied throughout the process are also classified into a proposed taxonomy.

Smart video surveillance is well suited for a broad range of applications. Moving object classification in the field of video surveillance is a key component of smart surveillance software. In this paper, we have proposed reliable software with its large features for people and object classification which works well in challenging real-world constraints, including the presence of shadows, low resolution imagery, occlusion, perspective distortions, arbitrary camera viewpoints, and groups of people. We have discussed a generic model of smart video surveillance systems that can meet requirements of strong commercial applications and also shown the implication of the software for the security purposes which made the whole system as a smart video surveillance. Smart surveillance systems use automatic image understanding techniques to extract information from the surveillance data and handling events and stored data efficiently.