Hidden Markov Model as a Framework for Situational Awareness

Lecture Notes in Computer Science, 2009

Situation Awareness (SA) is the problem of comprehending elements of an environment within a volume of time and space. It is a crucial factor in decision-making in dynamic environments. Current SA systems support the collection, filtering and presentation of data from different sources very well, and typically also some form of low-level data fusion and analysis, e.g., recognizing patterns over time. However, a still open research challenge is to build systems that support higher-level information fusion, viz., to integrate domain specific knowledge and automatically draw conclusions that would otherwise remain hidden or would have to be drawn by a human operator. To address this challenge, we have developed a novel system architecture that emphasizes the rôle of formal logic and automated theorem provers in its main components. Additionally, it features controlled natural language for operator I/O. It offers three logical languages to adequately model different aspects of the domain. This allows to build SA systems in a more declarative way than is possible with current approaches. From an automated reasoning perspective, the main challenges lay in combining (existing) automated reasoning techniques, from low-level data fusion of time-stamped data to semantic analysis and alert generation that is based on linear temporal logic. The system has been implemented and interfaces with Google-Earth to visualize the dynamics of situations and system output. It has been successfully tested on realistic data, but in this paper we focus on the system architecture and in particular on the interplay of the different reasoning components.

The detection of anomalies has a critical role in situational assessment. In this paper, we break down the concept of anomaly in the maritime domain into different levels and relate them to the JDL fusion model. We also show how uncertain context knowledge can be encoded through Markov Logic Networks (MLNs) which offer a convenient framework leveraging both the expressive power of first order logic and the probabilistic uncertainty management of Markov networks. Every formula in the knowledge base is assigned a weight indicating its confidence. Different types of knowledge with associated uncertainty can therefore be fused together within MLNs and on-line inference can be performed as input data is processed by the system, and the formulas are grounded in the knowledge base. Promising examples are demonstrated on a sample set of rules for maritime event and anomaly detection.

International Journal of Humanoid Robotics, 2008

Understanding intent is an important aspect of communication among people and is an essential component of the human cognitive system. This capability is particularly relevant for situations that involve collaboration among multiple agents or detection of situations that can pose a particular threat. In this paper, we propose an approach that allows a physical robot to detect intentions of others based on experience acquired through its own sensory-motor capabilities, then using this experience while taking the perspective of the agent whose intent should be recognized. Our method uses a novel formulation of Hidden Markov Models (HMMs) designed to model a robot's experience and interaction with the world when performing various actions. The robot's capability to observe and analyze the current scene employs a novel vision-based technique for target detection and tracking, using a non-parametric recursive modeling approach. We validate this architecture with a physically embedded robot, detecting the intent of several people performing various activities.

2014

Abstract. Situation Awareness (SA) is defined as the perception of elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future [1]. Lacking SA or having inadequate SA has been identified as one of the primary factors in accidents attributed to human error [2]. In this paper we present a probabilistic machine-learning-based approach for the real-time prediction of the focus of attention and deficits of SA using a Bayesian driver model as a driving monitor. This Bayesian driving monitor generates expectations conditional on the actions of the driver which are treated as evidence in the Bayesian driver model.

2005

Determining the situation within an environment is a key goal of smart environment research. A significant challenge in situation determination is reasoning about openended groups of people and devices that a smart environment may contain. Contemporary solutions are often tailored to the specific environment. In this position paper, we present a novel general situation determination framework, that by viewing people and tools as playing roles in a situation, can easily adapt recognition to incorporate the dynamic structure of a situation over time.

The change of focus in modern warfare from individual platforms to the network has caused a concomitant shift in supporting concepts and technologies. Greater emphasis is placed on interoperability and composeability. New technologies such as SOA and semantically aware systems have come into the spotlight. This paper argues that just as the problem space demands interoperability of diverse technologies, so must the solution space. In other words, not only are new approaches needed, but they must also come together as a seamlessly interoperable technological tool set. This can be accomplished only via a consistent multi-disciplinary approach. In this paper, we present some of the major requirements of today's Predictive Situation Awareness Systems (PSAW), propose our approach as a coordinated mix between state-of-the-art research efforts, and present the architecture for enabling our approach.

IFAC-PapersOnLine, 2017

This paper presents a novel approach to artificial situation awareness for an autonomous vehicle operating in complex dynamic environments populated by other agents. A key aspect of situation awareness is the use of mental models to predict future states of the environment, allowing safe and rational routing decisions to be made. We present a technique for predicting future discrete state transitions (such as the commencement of a turn) by other agents, based upon an uncertain mental model. Predictions take the form of univariate Gaussian Probability Density Functions which capture the inherent uncertainty in transition time whilst still providing great benefit to a decision making system. The prediction distributions are compared with Monte Carlo simulations and show an excellent correlation over long prediction horizons.

2010

To interact naturally with humans, robots needs to be aware of their own surroundings. This awareness is usually encoded in some implicit or explicit representation of the situated context. In this research report, we present a new framework for constructing rich belief models of the robot's environment.

ArXiv, 2021

The knowledge that an intelligent and autonomous mobile robot has and is able to acquire of itself and the environment, namely the situation, limits its reasoning, decision-making, and execution skills to efficiently and safely perform complex missions. Situational awareness is a basic capability of humans that has been deeply studied in fields like Psychology, Military, Aerospace, Education, etc., but it has barely been considered in robotics, which has focused on ideas such as sensing, perception, sensor fusion, state estimation, localization and mapping, spatial AI, etc. In our research, we connected the broad multidisciplinary existing knowledge on situational awareness with its counterpart in mobile robotics. In this paper, we survey the state-of-the-art robotics algorithms, we analyze the situational awareness aspects that have been covered by them, and we discuss their missing points. We found out that the existing robotics algorithms are still missing manifold important aspe...

The real-time manned-unmanned teaming of on-the-move Army assets will provide mobile commanders and warfighters with improved situation awareness from the sharing and fusion of heterogeneous distributed data sources information. Lockheed Martin Advanced Technology Laboratories (ATL) is improving situation awareness through three ATL-developed technologies: adaptive, modular, multisensor information fusion; Grapevine agent-based intelligent data dissemination; and agent-based data discovery. These technologies address the three key challenges to shared situation awareness: how to consume data, when to disseminate data, and where to access data. In particular, these technologies focus on the general task of real-time distributed heterogeneous information fusion in highly dynamic ad hoc networked environments. In this paper we describe the three technologies that enable real-time manned-unmanned teaming for shared situation awareness, as well as active research and development of new a...