Context and Activity Recognition

Cloud computing is an information technology model that provides access to system resources with higher level of services capability. These resources are considered reliable, flexible and affordable for many types of applications and users. Gaming industry is one filed that gained benefits of cloud computing as new cloud gaming architectures have been introduced. Many advantages of cloud gaming have affected the success of gaming according to the improvements on traditional online gaming. However, cloud gaming suffers from several drawbacks such as the huge amount of required video processing and the computational complexity needed. This paper shows the original system drawbacks and devises a new and novel algorithm for speeding up the encoding process and reducing the computational complexity. Improvements on the video codec led to 41% speeding up on the total encoding time with negligible loss of users’ satisfactions.

Semaphore, a way of communicating remotely, usually practiced in scouting activities. Information is delivered by gestures or movements using specific tools such as flags, paddles or rods. Teacher and instructors are needed for learning semaphore in conventional way as they will give examples and make correction when such an error occured. Based on the practical need to provide an alternative way to learn semaphore, this research proposes the use of geometry algorithm to develop a semaphore gesture recognition based on skeleton images that read from Kinect sensor. Euclidean distance and law cosines are two formulas that applied to generate gesture parameters of each alphabet. Recognition is achieved by comparing a pair of values of model and real-time gesture. Accuracy of this system that have been measured using RMSE with 30° of tolerance yields 90.76% for Alphabet and 88% for Word.

This paper presents a state-of-the-art wearable device integrated with a sophisticated human activity recognition algorithm designed for the accurate identification of human fitness activities. The wearable device includes a microcontroller, tri-axial sensor, Bluetooth transmission module, and power supply circuit. The accompanying activity recognition algorithm incorporates motion signal collection, signal preprocessing, flexible human motion detection, signal normalization, feature extraction, normalization, and reduction to recognize activities based on accelerations and angular velocities. The device is tested with 5 participants performing 6 common domestic activities in a laboratory environment, achieving reasonable recognition rates of 82.23% using the Recurrent Neural Network (RNN) classification algorithm. The successful validation of the device and algorithm highlights its effectiveness. However, some challenges remain, such as real-world performance outside the laboratory setting, user comfort, and power consumption. To address these, future iterations could focus on improving sensor precision, optimizing algorithm parameters, enhancing battery life, and refining the design for a more seamless user experience. By addressing these challenges, the technology could lead to even better solutions for human activity recognition in various practical applications.

As the world grapples with an increase in diseases including COVID-19, the Internet of Medical Things (IoMT) emerges as a complementary technology to the healthcare staff, which is constantly overburdened due to, among other things, a lack of staff. Untrained users' increased online presence exposes them to cyberattack threats. Authentication is the first line of defense for protecting medical data, but existing solutions are rigid as they do not consider the user's context and capabilities, making them unusable for some groups of users who eventually shun them. This paper proposes an adaptive user authentication framework based on Machine Learning that adapts to user profiles and context during a login attempt to determine the likelihood of the attempt being illegitimate before assigning an appropriate authentication mechanism. The proposed edge-centric framework uses a Naive Bayes classifier and CoFRA model to determine the risk associated with a login attempt based on biometric data from wearable sensors, non-biometric data from a smartphone, and some predefined data. User backgrounds and preferences were solicited, and results showed that users despite their ICTSkills, ages, jobs, and years of experience like to use simple physiological biometrics for authentication. An Android App was developed using the User-Centred design and installed on a smartphone which communicated with a PineTime smartwatch. Sensor data was used as input in calculating the risk associated with an access request to decide whether to authenticate, step up authentication, or block a request using rule and role-based access control techniques while also using biometric data for nonintrusive health monitoring. Once implemented, the framework is expected to improve user experience in authentication and promote the use of IoT in healthcare.

Many research efforts have been made on human context recognition, especially activity recognition using sensors such as accelerometers embedded in smartphones. However, few studies are conducted for recognizing human context while operating smartphones (smartphone operating context or socontext in short). Examples of so-contexts are smoking or eating while operating smartphones and playing a game while in the train. Estimating so-contexts will bring new services such as notification timing optimization and user interface optimization. In this paper, aiming to provide a tool toward so-context estimation, we propose a system that monitors, recognizes and outputs user's touch operations on Android phones. To realize so-context estimation, the system needs to satisfy three requirements: (1) it should work on any android device, (2) it should run in the background of any application, and (3) it should identify touch operations in high-level format and output the identified operation...

Mobile authentication/identification has grown into a priority issue nowadays because of its existing outdated mechanisms, such as PINs or passwords. In this paper, we introduce gait recognition by using a mobile accelerometer as not only effective but also as an implicit identification model. Unlike previous works, the gait recognition only performs well with a particular mobile specification (e.g., a fixed sampling rate). Our work focuses on constructing a unique adaptive mechanism that could be independently deployed with the specification of mobile devices. To do this, the impact of the sampling rate on the preprocessing steps, such as noise elimination, data segmentation, and feature extraction, is examined in depth. Moreover, the degrees of agreement between the gait features that were extracted from two different mobiles, including both the Average Error Rate (AER) and Intra-class Correlation Coefficients (ICC), are assessed to evaluate the possibility of constructing a device-independent mechanism. We achieved the classification accuracy approximately 91.33 ± 0.67 % for both devices, which showed that it is feasible and reliable to construct adaptive cross-device gait recognition on a mobile phone.

An outburst of Covid-19, a new disease by coronavirus has been noted by December 2019 in China and subsequently this Covid-19 spread throughout the world. The serious effect of this disease causes death due to the failure of respiratory system of human. Nowadays the spreading of the coronavirus is uncontrollable especially in crowded places as well as due to avoiding the protocols given by world health organization (WHO) to reduce the spread of Covid-19. Many researchers have involved in identifying vaccine for giving proper treatment to the affected people and to avoid further spreading. In addition, with this some other researchers have also involved in using modern technologies to prevent the spreading of Covid-19 as well as to identify the disease in very earlier stage and to reduce the rate of death. One of the modern systems recommended by the researchers is that smartwatch. By using smartwatches, the user's body temperature, heart rate and blood pressure can be measured, and the smartwatch can be automatically charged by body heat. This kind of smartwatches can be controlled by chatbot Google Assistant. This paper discusses the design, principle of operation and features of different smartwatches and their usage. Based on the analysis, it is identified that the availability of recent cost-effective developments of smartwatches are available on reducing spread of Covid-19.

The 3D CAD shapes in current 3D benchmarks are mostly collected from online model repositories. Thus, they typically have insufficient geometric details and less informative textures, making them less attractive for comprehensive and subtle research in areas such as high-quality 3D mesh and texture recovery. This paper presents 3D Furniture shape with TextURE (3D-FUTURE): a richly-annotated and large-scale repository of 3D furniture shapes in the household scenario. At the time of this technical report, 3D-FUTURE contains 20,240 clean and realistic synthetic images of 5,000 different rooms. There are 9,992 unique detailed 3D instances

I would like to dedicate these lines to express my gratitude to all the people who have helped me in a way or another to achieve this goal that is so important to me. First, I would like to thank my tutor and director, Dr. Ramón Barber, for introducing me to the exciting world of research, for all his support, guidance, trust, and patience over these years of my Ph.D. studies. Thank you for giving me the opportunity to work with you. I also thank Dr. Oscar Martínez, for all the advice and technical talks and for allowing me to visit his laboratory in Sweden. I would like to thank the University Carlos III of Madrid and all the Robotics Lab members I met during these years. Thank you for your support, professionalism, and friendship; It is an honor to belong to this great research group. I would like to especially thank my lab partner Clara Gómez. Thank you for being such a great partner and friend over these years, for your support, professionalism, your dedication at work. Thank you for being my partner in conferences and research visits. Definitely, the Ph.D. would not have been the same without you. I would like to thank Dr. Zoltan Marton and his team for giving me the opportunity to work with them at the DLR institute in Germany. I would also like to thank Prof. Robert Babuška for welcoming me to his lab at the CTU in Prague. Especially thanks to Erik Derner for being a great lab partner, for our interesting discussions, for his professionalism and friendship. I also thank the people I met at Örebro University, especially Eduardo Gutiérrez; thank you for your support and friendship. Finally and most importantly, I would like to thank my family. Thank you for all your support, your love, and for always being there. Despite the more than 7000 km. that separate us, I am always with you, and I know that you are with me celebrating this achievement. I leave for last, the person I must thank for having embarked on this path. I want to thank my husband, Jesús. Thank you for supporting me and helping me accomplish this goal, for being my partner on this roller coaster of life, for always believing in me, and for having a word of encouragement when I felt like I could not continue. Without you, I am sure I would not have made it. For you all my love and my gratitude. To all my sincere thanks.

Recent years have witnessed a huge progress in the automatic identification of individual primitives of human behavior, such as activities or locations. However, the complex nature of human behavior demands more abstract contextual information for its analysis. This work presents an ontology-based method that combines low-level primitives of behavior, namely activity, locations and emotions, unprecedented to date, to intelligently derive more meaningful high-level context information. The paper contributes with a new open ontology describing both low-level and high-level context information, as well as their relationships. Furthermore, a framework building on the developed ontology and reasoning models is presented and evaluated. The proposed method proves to be robust while identifying high-level contexts even in the event of erroneously-detected low-level contexts. Despite reasonable inference times being obtained for a relevant set of users and instances, additional work is requi...

Recent years have witnessed a huge progress in the automatic identification of individual primitives of human behavior, such as activities or locations. However, the complex nature of human behavior demands more abstract contextual information for its analysis. This work presents an ontology-based method that combines low-level primitives of behavior, namely activity, locations and emotions, unprecedented to date, to intelligently derive more meaningful high-level context information. The paper contributes with a new open ontology describing both low-level and high-level context information, as well as their relationships. Furthermore, a framework building on the developed ontology and reasoning models is presented and evaluated. The proposed method proves to be robust while identifying high-level contexts even in the event of erroneously-detected low-level contexts. Despite reasonable inference times being obtained for a relevant set of users and instances, additional work is requi...

Abstract—Context-aware application models can provide personalized services to users through user-centric integration of contexts. Recently, several research activities on context integration have been reported, However, the existing research activities don't consider much how contexts are integrated in a unified way. In this paper, we propose a unified method of user-centric integration of contexts for context-aware applications. The proposed method enables to extract the meaningful contexts based on each fusion procedure of ...

Recent years have witnessed a huge progress in the automatic identification of individual primitives of human behavior, such as activities or locations. However, the complex nature of human behavior demands more abstract contextual information for its analysis. This work presents an ontology-based method that combines low-level primitives of behavior, namely activity, locations and emotions, unprecedented to date, to intelligently derive more meaningful high-level context information. The paper contributes with a new open ontology describing both low-level and high-level context information, as well as their relationships. Furthermore, a framework building on the developed ontology and reasoning models is presented and evaluated. The proposed method proves to be robust while identifying high-level contexts even in the event of erroneously-detected low-level contexts. Despite reasonable inference times being obtained for a relevant set of users and instances, additional work is requi...

Recent years have witnessed a huge progress in the automatic identification of individual primitives of human behavior, such as activities or locations. However, the complex nature of human behavior demands more abstract contextual information for its analysis. This work presents an ontology-based method that combines low-level primitives of behavior, namely activity, locations and emotions, unprecedented to date, to intelligently derive more meaningful high-level context information. The paper contributes with a new open ontology describing both low-level and high-level context information, as well as their relationships. Furthermore, a framework building on the developed ontology and reasoning models is presented and evaluated. The proposed method proves to be robust while identifying high-level contexts even in the event of erroneously-detected low-level contexts. Despite reasonable inference times being obtained for a relevant set of users and instances, additional work is requi...

There is sufficient evidence proving the impact that negative lifestyle choices have on people's health and wellness. Changing unhealthy behaviours requires raising people's self-awareness and also providing healthcare experts with a thorough and continuous description of the user's conduct. Several monitoring techniques have been proposed in the past to track users' behaviour; however, these approaches are either subjective and prone to misreporting, such as questionnaires, or only focus on a specific component of context, such as activity counters. This work presents an innovative multimodal context mining framework to inspect and infer human behaviour in a more holistic fashion. The proposed approach extends beyond the state-of-the-art, since it not only explores a sole type of context, but also combines diverse levels of context in an integral manner. Namely, low-level contexts, including activities, emotions and locations, are identified from heterogeneous senso...

The taxi and passenger queue contexts indicate the various states of queues related to taxis and passengers (i.e. taxis are waiting for passengers, passengers are waiting for taxis, both are waiting for each other, none is waiting). Predicting these queue contexts in a future time is very important for better airport ground transport operations. However, queue context prediction at the airport is a challenging problem due to the presence of different contextual factors i.e., time, weather, taxi trips, flight arrivals and many more. Also these taxi and passenger queue contexts at the airport are imbalanced since some of the contexts are very infrequently occurring compared to others. In this paper, we address the problem of predicting imbalanced taxi and passenger queue contexts at the airport. First, we investigate different contextual factors, including time, taxi trips, passengers and weather for queue context prediction. Then we propose a detailed step by step solution to address...

h i g h l i g h t s • We use a classifier based on a novel fusion of feature vectors (the VFH-Texton). • We derive an object-to-object context MRF model based on Flickr label co-occurrence data. • We investigate the model's parameters' convergence as a function of Flickr's sample size. • We train the system on the RGB-D Object Dataset and test on the NYU Dataset as well. • Finally we illustrate an increasing performance through the use of the MRF.

We present a modified Temporal Deep Belief Networks (TDBN) for human motion analysis and synthesis by incorporating Sparse Encoding Symmetric Machines (SESM) improvement on its pre-training. SESM consisted of two important terms: regularization and sparsity. In this paper, we measure the effect of these two terms on the smoothness of synthesized (or generated) motion. The smoothness is measured as the standard deviation of five bones movements with three motion transitions. We also address how these two terms influence the free energy and reconstruction error profiles during pre-training of the Restricted Boltzmann Machines (RBM) layers and the Conditional RBM (CRBM) layers. For this purpose, we compare gait transitions by bifurcation experiments using four different TDBN settings: original TDBN; modified-TDBN(R): a TDBN with only regularization constraint; modified-TDBN(S): a TDBN with only sparsity constraint; and modified-TDBN(R+S): a TDBN with regularization plus sparsity constr...

h i g h l i g h t s • We use a classifier based on a novel fusion of feature vectors (the VFH-Texton). • We derive an object-to-object context MRF model based on Flickr label co-occurrence data. • We investigate the model's parameters' convergence as a function of Flickr's sample size. • We train the system on the RGB-D Object Dataset and test on the NYU Dataset as well. • Finally we illustrate an increasing performance through the use of the MRF.

h i g h l i g h t s • We use a classifier based on a novel fusion of feature vectors (the VFH-Texton). • We derive an object-to-object context MRF model based on Flickr label co-occurrence data. • We investigate the model's parameters' convergence as a function of Flickr's sample size. • We train the system on the RGB-D Object Dataset and test on the NYU Dataset as well. • Finally we illustrate an increasing performance through the use of the MRF.

This work presents the Mining Minds Context Ontology, an ontology for the identification of human behavior. This ontology comprehensively models high-level context based on low-level information, including the user activities, locations, and emotions. The Mining Minds Context Ontology is the means to infer high-level context from the lowlevel information. High-level contexts can be inferred from unclassified contexts by reasoning on the Mining Minds Context Ontology. The Mining Minds Context Ontology is shown to be flexible enough to operate in real life scenarios in which emotion recognition systems may not always be available. Furthermore, it is demonstrated that the activity and the location might not be enough to detect some of the high-level contexts, and that the emotion enables a more accurate high-level context identification. This work paves the path for the future implementation of the high-level context recognition system in the Mining Minds project.

The monitoring of human lifestyles has gained much attention in the recent years. This work presents a novel approach to combine multiple context-awareness technologies for the automatic analysis of people's conduct in a comprehensive and holistic manner. Activity recognition, emotion recognition, location detection, and social analysis techniques are integrated with ontological mechanisms as part of a framework to identify human behavior. Key architectural components, methods and evidences are described in this paper to illustrate the interest of the proposed approach.

Behavior-based continuous authentication is an increasingly popular methodology that utilizes behavior modeling and sensing for authentication and account access authorization. As an appearing behavioral biometric, user interaction patterns with mobile devices focus on verifying their identity in terms of their features or operating styles while interacting with devices. However, unimodal continuous authentication schemes, which are on the basis of a single source of interaction information, can only deal with a particular action or scenario. Hence, multimodal systems should be taken to suit for various environmental conditions especially in circumstances of attacks. In this paper, we propose a multimodal continuous authentication method both based on static interaction patterns and dynamic interaction patterns with mobile devices. Behavioral biometric features, HMHP, which is combined hand motion (HM) and hold posture (HP), are essentially established upon the touch screen and accelerator and capture the variation model of microhand motions and hold patterns generated in both dynamic and static scenes. By combining the features of HM and HP, the fusion feature HMHP achieves 97% accuracy with a 3.49% equal error rate.

There is a rapidly increasing amount of research on the use of brain activity patterns as a basis for biometric user verification. The vast majority of this research is based on Electroencephalogram (EEG), a technology which measures the electrical activity along the scalp. In this paper, we evaluate Functional Near-Infrared Spectroscopy (fNIRS) as an alternative approach to brain activity-based user authentication. fNIRS is centered around the measurement of light absorbed by blood and, compared to EEG, has a higher signal-to-noise ratio, is more suited for use during normal working conditions, and has a much higher spatial resolution which enables targeted measurements of specific brain regions. Based on a dataset of 50 users that was analysed using an SVM and a Naïve Bayes classifier, we show fNIRS to respectively give EERs of 0.036 and 0.046 when using our best channel configuration. Further, we present some results on the areas of the brain which demonstrated highest discriminative power. Our findings indicate that fNIRS has significant promise as a biometric authentication modality.

Object recognition approaches have recently been extended to yield, aside of the object class output, also viewpoint or pose. Training such approaches typically requires additional viewpoint or keypoint annotation in the training data or, alternatively, synthetic CAD models. In this paper, we present an approach that creates a dataset of images annotated with bounding boxes and viewpoint labels in a fully automated manner from videos. We assume that the scene is static in order to reconstruct 3D surfaces via structure from motion. We automatically detect when the reconstruction fails and normalize for the viewpoint of the 3D models by aligning the reconstructed point clouds. Exemplarily for cars we show that we can expand a large dataset of annotated single images and obtain improved performance when training a viewpoint regressor on this joined dataset.

Nowadays, mobile phones are not only used for mere communication such as calling or sending text messages. Mobile phones are becoming the main computer device in people's lives. Besides, thanks to the embedded sensors (Accelerometer, digital compass, gyroscope, GPS, and so on) is possible to improve the user experience. Activity recognition aims to recognize actions and goals of individual from a series of observations of themselves, in this case is used an accelerometer.

Nowadays, mobile phones are not only used for mere communication such as calling or sending text messages. Mobile phones are becoming the main computer device in people's lives. Besides, thanks to the embedded sensors (Accelerometer, digital compass, gyroscope, GPS, and so on) is possible to improve the user experience. Activity recognition aims to recognize actions and goals of individual from a series of observations of themselves, in this case is used an accelerometer.

Nowadays, mobile phones are not only used for mere communication such as calling or sending text messages. Mobile phones are becoming the main computer device in people's lives. Besides, thanks to the embedded sensors (Accelerometer, digital compass, gyroscope, GPS, and so on) is possible to improve the user experience. Activity recognition aims to recognize actions and goals of individual from a series of observations of themselves, in this case is used an accelerometer.

People identification using gait information (i.e., the way a person walks) obtained from inertial sensors is a robust approach that can be used in multiple situations where vision-based systems are not applicable. Typically, previous methods use hand-crafted features or deep learning approaches with pre-processed features as input. In contrast, we present a new deep learning-based end-to-end approach that employs raw inertial data as input. By this way, our approach is able to automatically learn the best representations without any constraint introduced by the pre-processed features. Moreover, we study the fusion of information from multiple inertial sensors and multi-task learning from multiple labels per sample. Our proposal is experimentally validated on the challenging dataset OU-ISIR, which is the largest available dataset for gait recognition using inertial information. After conducting an extensive set of experiments to obtain the best hyper-parameters, our approach is able...

For robots to be able to fluidly collaborate with and keep company to humans in indoor spaces, they need to be able to perceive and understand such environments, including furniture and rooms. Towards that goal, we present a system for indoor furniture and room recognition for robots, which has two significant novelties: it utilizes internet-derived as well as self-captured models for training, and also uses object- and room-context information mined through the internet, in order to bootstrap and enhance its performance. Thus, the system also acts as an example of how autonomous robot entities can benefit from utilizing online information and services. Many interesting subproblems, including the peculiarities of utilizing such online sources, are discussed, followed by a real- world empirical evaluation of the system, which shows highly promising results

Gait recognition is a technique that identifies or verifies people based upon their walking patterns. Smartwatches, which contain an accelerometer and gyroscope have recently been used to implement gait-based biometrics. However, this prior work relied upon data from single sessions for both training and testing, which is not realistic and can lead to overly optimistic performance results. This paper aims to remedy some of these problems by training and evaluating a smartwatch-based biometric system on data obtained from different days. Also, it proposes an advanced feature selection approach to identify optimal features for each user. Two experiments are presented under three different scenarios: Same-Day, Mixed-Day, and Cross-Day. Competitive results were achieved (best EERs of 0.13% and 3.12% by using the Same day data for accelerometer and gyroscope respectively and 0.69% and 7.97% for the same sensors under the Cross-Day evaluation. The results show that the technology is sufficiently capable and the signals captured sufficiently discriminative to be useful in performing gait recognition.

A secure, user-convenient approach to authenticate users on their mobile devices is required as current approaches (e.g., PIN or Password) suffer from security and usability issues. Transparent Authentication Systems (TAS) have been introduced to improve the level of security as well as offer continuous and unobtrusive authentication (i.e., user friendly) by using various behavioural biometric techniques. This paper presents the usefulness of using smartwatch motion sensors (i.e., accelerometer and gyroscope) to perform Activity Recognition for the use within a TAS. Whilst previous research in TAS has focused upon its application in computers and mobile devices, little attention is given to the use of wearable devices-which tend to be sensor-rich highly personal technologies. This paper presents a thorough analysis of the current state of the art in transparent and continuous authentication using acceleration and gyroscope sensors and a technology evaluation to determine the basis for such an approach. The best results are average Euclidean distance scores of 5.5 and 11.9 for users' intra acceleration and gyroscope signals respectively and 24.27 and 101.18 for users' inter acceleration and gyroscope activities accordingly. The findings demonstrate that the technology is sufficiently capable and the nature of the signals captured sufficiently discriminative to be useful in performing Activity Recognition.

Many research efforts have been made on human context recognition, especially activity recognition using sensors such as accelerometers embedded in smartphones. However, few studies are conducted for recognizing human context while operating smartphones (smartphone operating context or so-context in short). Examples of so-contexts are smoking or eating while operating smartphones and playing a game while in the train. Estimating so-contexts will bring new services such as notification timing optimization and user interface optimization. In this paper, aiming to provide a tool toward so-context estimation, we propose a system that monitors, recognizes and outputs user's touch operations on Android phones. To realize so-context estimation, the system needs to satisfy three requirements: (1) it should work on any android device, (2) it should run in the background of any application, and (3) it should identify touch operations in high-level format and output the identified operations with the detailed information like finger position and movement for so-context recognition. For the above requirements, we developed our proposed system as an Android application which analyzes raw data output by the OS including a time series of points on the screen, and recognizes 7 representative high-level touch operations such as swipe and rotate with information on the number of fingers used, the pressure level and the track between the start-point and the end-point. We evaluated our system and confirmed that it achieved recognition accuracies of 100% for single-or double-finger swipe and single-finger touch operations (swipe), and 98% for two-finger touch operations (pinch, rotate, etc.). Moreover, to show the applicability of our system, we tried to recognize the phone holding style as a so-context from touch operations. As a result, we confirmed that it achieved F-measure of 96.5% for classification among 8 different holding styles.

— In recent years, healthcare and wellness platforms are developed rapidly with the advent of smart devices which possess diverse sensors. Existing systems are limited to provide simple health status visualization services from single device or single sensor, which make them unable to provide timely high quality services. This paper proposes Human Centric Awareness Framework based on diverse IoT devices to provide high quality services timely.

—The combination of ontology based context-awareness and machine learning context classification is an interesting research area. The determined contexts are obtained using semantic reasoning based on context ontology developed by expert using domain specific rules. This reasoning suffer challenges of soundness and completeness in real-time deployment. This paper addresses the aforementioned challenges from semantic reasoning by embracing machine learning modeling and classification benefits. Machine learning relies on data, for this we developed training and deployment phase for ontological ABox assertions. Approximately 99.99% precision through machine learning approach was achieved over 91.5% accuracy with semantic reasoning. The statistical evaluation proves the improvement in terms of accuracy for context prediction and overall performance.

The emerging research on automatic identification of user's contexts from the cross-domain environment in ubiquitous and pervasive computing systems has proved to be successful. Monitoring the diversified user's contexts and behaviors can help in controlling lifestyle associated to chronic diseases using context-aware applications. However, availability of cross-domain heterogeneous contexts provides a challenging opportunity for their fusion to obtain abstract information for further analysis. This work demonstrates extension of our previous work from a single domain (i.e., physical activity) to multiple domains (physical activity, nutrition and clinical) for context-awareness. We propose multi-level Context-aware Framework (mlCAF), which fuses the multi-level cross-domain contexts in order to arbitrate richer behavioral contexts. This work explicitly focuses on key challenges linked to multi-level context modeling, reasoning and fusioning based on the mlCAF open-source ontology. More specifically, it addresses the interpretation of contexts from three different domains, their fusioning conforming to richer contextual information. This paper contributes in terms of ontology evolution with additional domains, context definitions, rules and inclusion of semantic queries. For the framework evaluation, multi-level cross-domain contexts collected from 20 users were used to ascertain abstract contexts, which served as basis for behavior modeling and lifestyle identification. The experimental results indicate a context recognition average accuracy of around 92.65% for the collected cross-domain contexts.

Recent years have witnessed a huge progress in the automatic identification of individual primitives of human behavior, such as activities or locations. However, the complex nature of human behavior demands more abstract contextual information for its analysis. This work presents an ontology-based method that combines low-level primitives of behavior, namely activity, locations and emotions, unprecedented to date, to intelligently derive more meaningful high-level context information. The paper contributes with a new open ontology describing both low-level and high-level context information, as well as their relationships. Furthermore, a framework building on the developed ontology and reasoning models is presented and evaluated. The proposed method proves to be robust while identifying high-level contexts even in the event of erroneously-detected low-level contexts. Despite reasonable inference times being obtained for a relevant set of users and instances, additional work is required to scale to long-term scenarios with a large number of users.

There is sufficient evidence proving the impact that negative lifestyle choices have on people's health and wellness. Changing unhealthy behaviours requires raising people's self-awareness and also providing healthcare experts with a thorough and continuous description of the user's conduct. Several monitoring techniques have been proposed in the past to track users' behaviour; however, these approaches are either subjective and prone to misreporting, such as questionnaires, or only focus on a specific component of context, such as activity counters. This work presents an innovative multimodal context mining framework to inspect and infer human behaviour in a more holistic fashion. The proposed approach extends beyond the state-of-the-art, since it not only explores a sole type of context, but also combines diverse levels of context in an integral manner. Namely, low-level contexts, including activities, emotions and locations, are identified from heterogeneous sensory data through machine learning techniques. Low-level contexts are combined using ontological mechanisms to derive a more abstract representation of the user's context, here referred to as high-level context. An initial implementation of the proposed framework supporting real-time context identification is also presented. The developed system is evaluated for various realistic scenarios making use of a novel multimodal context open dataset and data on-the-go, demonstrating prominent context-aware capabilities at both low and high levels. The last World Health Organization (WHO) global status report on noncommunicable diseases reveals that illnesses associated with lifestyle choices are currently the leading cause of death worldwide [1]. As a matter of fact, non-communicable diseases are responsible for more than two-thirds of the world's deaths, with more than 40% of them representing premature deaths under the age of 70 years. Recognizing this seriously worrying epidemic scenario, the WHO has defined a clear roadmap to alter the course of the so-called " slow-moving public health disaster ". Most of the policies

Modern digital technologies are paving the path to a revolutionary new concept of health and wellness care. Nowadays, many new solutions are being released and put at the reach of most consumers for promoting their health and wellness self-management. However, most of these applications are of very limited use, arguable accuracy and scarce interoperability with other similar systems. Accordingly, frameworks that may orchestrate, and intelligently leverage, all the data, information and knowledge generated through these systems are particularly required. This work introduces Mining Minds, an innovative framework that builds on some of the most prominent modern digital technologies, such as Big Data, Cloud Computing, and Internet of Things, to enable the provision of personalized healthcare and wellness support. This paper aims at describing the efficient and rational combination and interoperation of these technologies, as well as their integration with current and future personalized health and wellness services and business.

Spatial relationships between objects provide important information for text-based image retrieval. As users are more likely to describe a scene from a real world perspective, using 3D spatial relationships rather than 2D relationships that assume a particular viewing direction, one of the main challenges is to infer the 3D structure that bridges images with users' text descriptions. However, direct inference of 3D structure from images requires learning from large scale annotated data. Since interactions between objects can be reduced to a limited set of atomic spatial relations in 3D, we study the possibility of inferring 3D structure from a text description rather than an image, applying physical relation models to synthesize holistic 3D abstract object layouts satisfying the spatial constraints present in a textual description. We present a generic framework for retrieving images from a textual description of a scene by matching images with these generated abstract object layouts. Images are ranked by matching object detection outputs (bounding boxes) to 2D layout candidates (also represented by bounding boxes) which are obtained by projecting the 3D scenes with sampled camera directions. We validate our approach using public indoor scene datasets and show that our method outperforms an object occurrence based and a learned 2D pairwise relation based baselines.

This work presents the Mining Minds Context Ontology, an ontology for the identification of human behavior. This ontology comprehensively models high-level context based on low-level information, including the user activities, locations, and emotions. The Mining Minds Context Ontology is the means to infer high-level context from the low-level information. High-level contexts can be inferred from unclassified contexts by reasoning on the Mining Minds Context Ontology. The Mining Minds Context Ontology is shown to be flexible enough to operate in real life scenarios in which emotion recognition systems may not always be available. Furthermore, it is demonstrated that the activity and the location might not be enough to detect some of the high-level contexts, and that the emotion enables a more accurate high-level context identification. This work paves the path for the future implementation of the high-level context recognition system in the Mining Minds project.

For robots to be able to fluidly collaborate with and keep company to humans in indoor spaces, they need to be able to perceive and understand such environments, including furniture and rooms. Towards that goal, we present a system for indoor furniture and room recognition for robots, which has two significant novelties: it utilizes internet-derived as well as self-captured models for training, and also uses object-and room-context information mined through the internet, in order to bootstrap and enhance its performance. Thus, the system also acts as an example of how autonomous robot entities can benefit from utilizing online information and services. Many interesting subproblems, including the peculiarities of utilizing such online sources, are discussed, followed by a realworld empirical evaluation of the system, which shows highly promising results.

Recent advances in computer vision on the one hand, and imaging technologies on the other hand, have opened up a number of interesting possibilities for robust 3-D scene labeling. This paper presents contributions in several directions to improve the state-of-the-art in RGB-D scene labeling. First, we present a novel combination of depth and color features to recognize different object categories in isolation. Then, we use a context model that exploits detection results of other objects in the scene to jointly optimize labels of co-occurring objects in the scene. Finally, we investigate the use of social media mining to develop the context model, and provide an investigation of its convergence. We perform thorough experimentation on both the publicly available RGB-D Dataset from the University of Washington as well as on the NYU scene dataset. An analysis of the results shows interesting insights about contextual object category recognition, and its benefits. Figure 1: Proposed system architecture for context-aware object category recognition

For robots to be able to fluidly collaborate with and keep company to humans in indoor spaces, they need to be able to perceive and understand such environments, including furniture and rooms. Towards that goal, we present a system for indoor furniture and room recognition for robots, which has two significant novelties: it utilizes internet-derived as well as self-captured models for training, and also uses object-and room-context information mined through the internet, in order to bootstrap and enhance its performance. Thus, the system also acts as an example of how autonomous robot entities can benefit from utilizing online information and services. Many interesting subproblems, including the peculiarities of utilizing such online sources, are discussed, followed by a realworld empirical evaluation of the system, which shows highly promising results.

Recent advances in computer vision on the one hand, and imaging technologies on the other hand, have opened up a number of interesting possibilities for robust 3-D scene labeling. This paper presents contributions in several directions to improve the state-of-the-art in RGB-D scene labeling. First, we present a novel combination of depth and color features to recognize different object categories in isolation. Then, we use a context model that exploits detection results of other objects in the scene to jointly optimize labels of co-occurring objects in the scene. Finally, we investigate the use of social media mining to develop the context model, and provide an investigation of its convergence. We perform thorough experimentation on both the publicly available RGB-D Dataset from the University of Washington as well as on the NYU scene dataset. An analysis of the results shows interesting insights about contextual object category recognition, and its benefits. Figure 1: Proposed system architecture for context-aware object category recognition

The collective intelligence that emerges from the collaboration, competition, and co-ordination among individuals in social networks has opened up new opportunities for knowledge extraction. Valuable knowledge is stored and often "hidden" in massive user contributions, challenging researchers to find methods for leveraging these contributions and unfold this knowledge. In this chapter we investigate the problem of knowledge extraction from social media. We provide background information for knowledge extraction methods that operate on social media, and present three methods that use Flickr data to extract different types of knowledge namely, the community structure of tag-networks, the emerging trends and events in users tag activity, and the associations between image regions and

We present a modified Temporal Deep Belief Networks (TDBN) for human motion analysis and synthesis by incorporating Sparse Encoding Symmetric Machines (SESM) improvement on its pre-training. SESM consisted of two important terms: regularization and sparsity. In this paper, we measure the effect of these two terms on the smoothness of synthesized (or generated) motion. The smoothness is measured as the standard deviation of five bones movements with three motion transitions. We also address how these two terms influence the free energy and reconstruction error profiles during pre-training of the Restricted Boltzmann Machines (RBM) layers and the Conditional RBM (CRBM) layers. For this purpose, we compare gait transitions by bifurcation experiments using four different TDBN settings: original TDBN; modified-TDBN(R): a TDBN with only regularization constraint; modified-TDBN(S): a TDBN with only sparsity constraint; and modified-TDBN(R+S): a TDBN with regularization plus sparsity constraints. These experiments shows that the modified-TDBN(R+S) reaches lower energy faster in RBM pre-training and reach lower reconstruction error in the CRBM training. Even though the smoothness of the synthesized motion from the modified-TDBN approaches is slightly less smooth than the original TDBN, they are more responsive to the action command to change a motion (from run to walk or vice versa) while preserving the smoothness during motion transitions without incurring much overhead computation time.

This paper presents a novel 3-D object recognition
framework for a service robot to eliminate false detections
in cluttered office environments where objects are in a great
diversity of shapes and difficult to be represented by exact models.
Laser point clouds are first converted to bearing angle images
and a Gentleboost-based approach is then deployed for multiclass
object detection. In order to solve the problem of variable
object scales in object detection, a scale coordination technique
is adopted in every subscene that is segmented from the whole
scene according to the spatial distribution of 3-D laser points.
Moreover, semantic information (e.g., ceilings, floors, and walls)
extracted from raw 3-D laser points is utilized to eliminate false
object detection results. K-means clustering and Mahalanobis
distance are finally deployed to perform object segmentation in
a 3-D laser point cloud accurately. Experiments were conducted
on a real mobile robot to show the validity and performance of
the proposed method.

With the rapid development of mobile devices and wireless technologies, a large number of mobile social network systems have emerged in the last few years. The migration of social networks from web-based applications onto mobile platforms not only increases the connectivity of people, but also promotes the convenience of people's life. Integrated with location-based information services, mobile social networks can help users search for a friend or a friend of a friend nearby, or glean some location-related information. This paper studies some current popular mobile social network systems,namely, Dogeball, Twitter, and Jaiku, analyzes their services, architectures, technologies used to provide location-based information, interaction methods, and other issues. Mobile social network systems have many advantages over traditional web-based social network systems, but they also face some challenges, such as business issues, privacy protection, and correct use.