Evidential event inference in transport video surveillance

Image and Vision Computing, 2009

In this paper, we propose a technique for detecting possible events in outdoor areas monitored by a video surveillance system. In particular, here we focus on the time spent by an object to carry out simple events. To have a statistical representation of the times commonly required to perform certain activities, mixtures of Gaussians are maintained for each event type. Such statistics are then exploited both for the analysis of the simple activities and for discovering anomalous situations (i.e. complex events). In these cases, the system requires the attention of the human operator. A novel way of presenting results to the operator is also discussed. Experiments have been performed on a multi-camera system for parking lot security.

This paper investigates the application of an inductive logic programming system, allied with Markov Logic Networks (MLNs), to the task of learning event models from large video datasets. A learning from interpretations setting is used to learn event models efficiently, these models define the structure of a MLN. The network parameters are obtained from discriminative learning and probabilistic inference is used to query the MLN for event recognition.

IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2000

Understanding Video Events, the translation of low-level content in video sequences into highlevel semantic concepts, is a research topic that has received much interest in recent years. Important applications of this work include smart surveillance systems, semantic video database indexing, and interactive systems. This technology can be applied to several video domains including: airport terminal, parking lot, traffic, subway stations, aerial surveillance, and sign language data. In this work we survey the two main components of the event understanding process: Abstraction and Event modeling. Abstraction is the process of molding the data into informative units to be used as input to the event model. Event modeling is devoted to describing events of interest formally and enabling recognition of these events as they occur in the video sequence. Event modeling can be further decomposed in the categories of Pattern Recognition Methods, State Event Models, and Semantic Event Models. In this survey we discuss this proposed taxonomy of the literature, offer a unifying terminology, and discuss popular abstraction schemes (e.g. Motion History Images) and event modeling formalisms (e.g. Hidden Markov Model) and their use in video event understanding using extensive examples from the literature. Finally we consider the application domain of video event understanding in light of the proposed taxonomy, and propose future directions for research in this field. 1

IEEE International Conference on Networking, Sensing and Control, 2004, 2004

We propose in this paper an approach for recognising either isolated individual, group of people or crowd behaviours in the context of visual surveillance of metro scenes using multiple cameras. In this context, a behaviour recognition module relies on a vision module composed of three tasks: (a) motion detection and frame to frame tracking, (b) multiple cameras combination and (c) long term tracking of individuals, groups of people and crowd evolving in the scene. For each tracked actor, the behaviour recognition module performs three levels of reasoning: states, events and scenarios. We have also defined a general framework to easily combine and tune various recognition methods (e.g. automaton, Bayesian network or AND/OR tree) dedicated to the analysis of specific situations (e.g. mono/multi actors activities, numerical/symbolic actions or temporal scenarios). Validation results on different methods used to recognise specific behaviours are described.

Proceedings of the 15th ACM on International conference on multimodal interaction - ICMI '13, 2013

We present a general framework for complex event recognition that is well-suited for integrating information that varies widely in detail and granularity. Consider the scenario of an agent in an instrumented space performing a complex task while describing what he is doing in a natural manner. The system takes in a variety of information, including objects and gestures recognized by RGB-D and descriptions of events extracted from recognized and parsed speech. The system outputs a complete reconstruction of the agent's plan, explaining actions in terms of more complex activities and filling in unobserved but necessary events. We show how to use Markov Logic (a probabilistic extension of first-order logic) to create a model in which observations can be partial, noisy, and refer to future or temporally ambiguous events; complex events are composed from simpler events in a manner that exposes their structure for inference and learning; and uncertainty is handled in a sound probabilistic manner. We demonstrate the effectiveness of the approach for tracking kitchen activities in the presence of noisy and incomplete observations.

2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06), 2006

We present a novel method for jointly performing recognition of complex events and linking fragmented tracks into coherent, long-duration tracks. Many event recognition methods require highly accurate tracking, and may fail when tracks corresponding to event actors are fragmented or partially missing. However, these conditions occur frequently from occlusions, traffic and tracking errors. Recently, methods have been proposed for linking track fragments from multiple objects under these difficult conditions. Here, we develop a method for solving these two problems jointly. A hypothesized event model, represented as a Dynamic Bayes Net, supplies data-driven constraints on the likelihood of proposed track fragment matches. These event-guided constraints are combined with appearance and kinematic constraints used in the previous track linking formulation. The result is the most likely track linking solution given the event model, and the highest event score given all of the track fragments. The event model with the highest score is determined to have occurred, if the score exceeds a threshold. Results demonstrated on a busy scene of airplane servicing activities, where many non-event movers and long fragmented tracks are present, show the promise of the approach to solving the joint problem.

IET Computer Vision, 2008

The authors propose a high-level scenario recognition algorithm for video sequence interpretation. The recognition of scenarios is based on a Bayesian networks approach. The model of a scenario contains two main layers. The first one allows events from the observed visual features to be highlighted and the second layer is focused on the temporal reasoning stage. The temporal layer uses specific nodes permitting an event-based approach. These nodes focus on the lifetime of events highlighted from the results of the first layer. The temporal layer then estimates the qualitative and quantitative relations between the different temporal events helpful for the recognition task. The global recognition algorithm is illustrated over real indoor image sequences of an abandoned baggage scenario.

2010

CCTV and sensor based surveillance systems are part of our daily lives now in this modern society due to the advances in telecommunications technology and the demand for better security. The analysis of sensor data produces semantic rich events describing activities and behaviours of objects being monitored. Three issues usually are associated with events descriptions. First, data could be collected from multiple sources (e.g., sensors, CCTVs, speedometers, etc). Second, descriptions about these data can be poor, inaccurate or uncertain when they are gathered from unreliable sensors or generated by analysis non-perfect algorithms. Third, in such systems, there is a need to incorporate domain specific knowledge, e.g., criminal statistics about certain areas or patterns, when making inferences. However, in the literature, these three phenomena are seldom considered in CCTV-based event composition models. To overcome these weaknesses, in this paper, we propose a general event modelling and reasoning model which can represent and reason with events from multiple sources including domain knowledge, integrating the Dempster-Shafer theory for dealing with uncertainty and incompleteness. We introduce a notion called event cluster to represent uncertain and incomplete events induced from an observation. Event clusters are then used in the merging and inference process. Furthermore, we provide a method to calculate the mass values of events which use evidential mapping techniques.

2014

Research in the video surveillance is gaining more popularity due to its widespread applications as well as social impact. In this paper, we present an intelligent framework for detection of multiple events in surveillance videos. Based on the principle of compositionality, we modularize the surveillance problems into a set of variables comprising regions-of-interest, classes (i.e. human, vehicle), attributes (i.e. speed, locality) and a set of notions (i.e. rules) associated to each of the attributes to construct a knowledge- based understanding of the environment. The final output from the reasoning process, which combines the definition domains of the various variables, allows a broader and integrated understanding of com- plex pattern of activities in the scene. This is in contrast to the state-of-the-art solutions that are only able to perform only a singular task, at a time. Experimental results on both the public and real-time datasets have demonstrated the effectiveness and robustness of the proposed framework in detecting multiple events in surveillance videos.

ACM Transactions on Computational Logic, 2015

Symbolic event recognition systems have been successfully applied to a variety of application domains, extracting useful information in the form of events, allowing experts or other systems to monitor and respond when significant events are recognised. In a typical event recognition application, however, these systems often have to deal with a significant amount of uncertainty. In this paper, we address the issue of uncertainty in logic-based event recognition by extending the Event Calculus with probabilistic reasoning. Markov Logic Networks are a natural candidate for our logic-based formalism. However, the temporal semantics of the Event Calculus introduce a number of challenges for the proposed model. We show how and under what assumptions we can overcome these problems. Additionally, we study how probabilistic modelling changes the behaviour of the formalism, affecting its key property, the inertia of fluents. Furthermore, we demonstrate the advantages of the probabilistic Event Calculus through examples and experiments in the domain of activity recognition, using a publicly available dataset for video surveillance.