Urban activities, particularly vehicle traffic, are contributing significantly to environmental p... more Urban activities, particularly vehicle traffic, are contributing significantly to environmental pollution with detrimental effects on public health. The ability to anticipate air quality in advance is critical for public authorities and the general public to plan and manage these activities, which ultimately help in minimizing the adverse impact on the environment and public health effectively. Thanks to recent advancements in Artificial Intelligence and sensor technology, forecasting air quality is possible through the consideration of various environmental factors. This paper presents our novel solution for air quality prediction and its correlation with different environmental factors and urban activities, such as traffic density. To this aim, we propose a multi-modal framework by integrating real-time data from different environmental sensors and traffic density extracted from Closed Circuit Television footage. The framework effectively addresses data inconsistencies arising from sensor and camera malfunctions within a streaming dataset. The dataset exhibits real-world complexities, including abrupt camera or station activations/deactivations, noise interference, and outliers. The proposed system tackles the challenge of predicting air quality at locations having no sensors or experiencing sensor failures by training a joint model on the data obtained from nearby stations/sensors using a Particle Swarm Optimization (PSO)-based merit fusion of the sensor data. The proposed methodology is evaluated using various variants of the LSTM model including Bi-directional LSTM, CNN-LSTM, and Convolutions LSTM (ConvLSTM) obtaining an improvement of 48%, 67%, and 173% for short-term, medium-term, and long-term periods, respectively, over the ARIMA model. Since the first industrial revolution in the 18th century, the planet's environment has experienced ongoing devastation due to factory emissions and an increase in urban activities, such as vehicle traffic, mining, and farming. As a result, various pollutants are increasingly released into the environment. Air quality is one of the major concerns resulting from the deterioration of the environment and the release of pollutants. The air could be polluted by various contaminants, such as Particulate Matter (PM1.0), PM2.5, PM10, CO, NO 2 , and SO 2 , as determined by the US Environmental Protection Agency (EPA) 1 . Each of these pollutants is emitted into the environment due to several factors, such as traffic, industrial waste, and gaseous emissions from homes and factories. Additionally, NO 2 from agricultural waste is also a significant contributor to air pollution. Poor air quality has a direct impact on people's health, resulting in various illnesses, such as lung diseases, asthma, cancer, and even fatalities 2 . According to the World Health Organization (WHO) 3 , around 99% of the global population breathes air exceeding WHO guideline limits and contains high levels of pollutants where the highest exposure is observed in low-and middle-income and developing countries. According to the UN Environment program 4 , the World Health Organization's (WHO) air quality guidelines are necessary to avoid the impacts of bad air, which causes around 7 million premature deaths per year. Policies and efforts to reduce air pollution could significantly improve air quality leading to improved climate and public health that will ultimately reduce the burden on the economy of low and middle-income countries. Real-time monitoring and forecasting of air quality are critical to remediation by keeping the authorities informed about the air quality to take necessary precautions and make informed decisions 5 . A system able to predict air pollution in the short, medium, and long term period and learn the correlation between urban activities and air pollution could greatly help in policy-making decisions. This correlation is critical as the deployment
Several membership inference (MI) attacks have been proposed to audit a target DNN. Given a set o... more Several membership inference (MI) attacks have been proposed to audit a target DNN. Given a set of subjects, MI attacks tell which subjects the target DNN has seen during training. This work focuses on the post-training MI attacks emphasizing high confidence membership detection-True Positive Rates (TPR) at low False Positive Rates (FPR). Current works in this category-likelihood ratio attack (LiRA) and enhanced MI attack (EMIA)-only perform well on complex datasets (e.g., CIFAR-10 and Imagenet) where the target DNN overfits its train set, but perform poorly on simpler datasets (0% TPR by both attacks on Fashion-MNIST, 2% and 0% TPR respectively by LiRA and EMIA on MNIST at 1% FPR). To address this, firstly, we unify current MI attacks by presenting a framework divided into three stages-preparation, indication and decision. Secondly, we utilize the framework to propose two novel attacks: (1) Adversarial Membership Inference Attack (AMIA) efficiently utilizes the membership and the non-membership information of the subjects while adversarially minimizing a novel loss function, achieving 6% TPR on both Fashion-MNIST and MNIST datasets; and (2) Enhanced AMIA (E-AMIA) combines EMIA and AMIA to achieve 8% and 4% TPRs on Fashion-MNIST and MNIST datasets respectively, at 1% FPR. Thirdly, we introduce two novel augmented indicators that positively leverage the loss information in the Gaussian neighborhood of a subject. This improves TPR of all four attacks on average by 2.5% and 0.25% respectively on Fashion-MNIST and MNIST datasets at 1% FPR. Finally, we propose simple, yet novel, evaluation metric, the running TPR average (RTA) at a given FPR, that better distinguishes different MI attacks in the low FPR region. We also show that AMIA and E-AMIA are more transferable to the unknown DNNs (other than the target DNN) and are more robust to DP-SGD training as compared to LiRA and EMIA.
The Visual Sentiment Analysis task is being offered for the first time at MediaEval. The main pur... more The Visual Sentiment Analysis task is being offered for the first time at MediaEval. The main purpose of the task is to predict the emotional response to images of natural disasters shared on social media. Disaster-related images are generally complex and often evoke an emotional response, making them an ideal use case of visual sentiment analysis. We believe being able to perform meaningful analysis of natural disaster-related data could be of great societal importance, and a joint effort in this regard can open several interesting directions for future research. The task is composed of three sub-tasks, each aiming to explore a different aspect of the challenge. In this paper, we provide a detailed overview of the task, the general motivation of the task, and an overview of the dataset and the metrics to be used for the evaluation of the proposed solutions.
This paper describes and evaluates the use of Generative Adversarial Networks (GANs) for path pla... more This paper describes and evaluates the use of Generative Adversarial Networks (GANs) for path planning in support of smart mobility applications such as indoor and outdoor navigation applications, individualized wayfinding for people with disabilities (e.g., vision impairments, physical disabilities, etc.), path planning for evacuations, robotic navigations, and path planning for autonomous vehicles. We propose an architecture based on GANs to recommend accurate and reliable paths for navigation applications. The proposed system can use crowdsourced data to learn the trajectories and infer new ones. The system provides users with generated paths that help them navigate from their local environment to reach a desired location. As a use case, we experimented with the proposed method in support of a wayfinding application in an indoor environment. Our experiments assert that the generated paths are correct and reliable. The accuracy of the classification task for the generated paths is up to 99% and the quality of the generated paths has a mean opinion score of 89%.
Social media have been widely exploited to detect and gather relevant information about opinions ... more Social media have been widely exploited to detect and gather relevant information about opinions and events. However, the relevance of the information is very subjective and rather depends on the application and the end-users. In this article, we tackle a specific facet of social media data processing, namely the sentiment analysis of disaster-related images by considering people's opinions, attitudes, feelings and emotions. We analyze how visual sentiment analysis can improve the results for the end-users/beneficiaries in terms of mining information from social media. We also identify the challenges and related applications, which could help defining a benchmark for future research efforts in visual sentiment analysis.
The development of smart cities and their fast-paced deployment is resulting in the generation of... more The development of smart cities and their fast-paced deployment is resulting in the generation of large quantities of data at unprecedented rates. Unfortunately, most of the generated data is wasted without extracting potentially useful information and knowledge because of the lack of established mechanisms and standards that benefit from the availability of such data. Moreover, the high dynamical nature of smart cities calls for new generation of machine learning approaches that are flexible and adaptable to cope with the dynamicity of data to perform analytics and learn from real-time data. In this article, we shed the light on the challenge of under utilizing the big data generated by smart cities from a machine learning perspective. Especially, we present the phenomenon of wasting unlabeled data. We argue that semi-supervision is a must for smart city to address this challenge. We also propose a three-level learning framework for smart cities that matches the hierarchical nature of big data generated by smart cities with a goal of providing different levels of knowledge abstractions. The proposed framework is scalable to meet the needs of smart city services. Fundamentally, the framework benefits from semi-supervised deep reinforcement learning where a small amount of data that has users' feedback serves as labeled data while a larger amount is without such users' feedback serves as unlabeled data. The framework utilizes a mix of labeled and unlabeled data to converge toward better control policies instead of wasting the unlabeled data. This paper also explores how deep reinforcement learning and its shift toward semi-supervision can handle the cognitive side of smart city services and improve their performance by providing several use cases spanning the different domains of smart cities. We also highlight several challenges as well as promising future research directions for incorporating machine learning and highlevel intelligence into smart city services.
Smart services are an important element of the smart cities and the Internet of Things (IoT) ecos... more Smart services are an important element of the smart cities and the Internet of Things (IoT) ecosystems where the intelligence behind the services is obtained and improved through the sensory data. Providing a large amount of training data is not always feasible; therefore, we need to consider alternative ways that incorporate unlabeled data as well. In recent years, Deep reinforcement learning (DRL) has gained great success in several application domains. It is an applicable method for IoT and smart city scenarios where auto-generated data can be partially labeled by users' feedback for training purposes. In this paper, we propose a semi-supervised deep reinforcement learning model that fits smart city applications as it consumes both labeled and unlabeled data to improve the performance and accuracy of the learning agent. The model utilizes Variational Autoencoders (VAE) as the inference engine for generalizing optimal policies. To the best of our knowledge, the proposed model is the first investigation that extends deep reinforcement learning to the semi-supervised paradigm. As a case study of smart city applications, we focus on smart buildings and apply the proposed model to the problem of indoor localization based on BLE signal strength. Indoor localization is the main component of smart city services since people spend significant time in indoor environments. Our model learns the best action policies that lead to a close estimation of the target locations with an improvement of 23% in terms of distance to the target and at least 67% more received rewards compared to the supervised DRL model.
While the technique of Deep Neural Networks (DNNs) has been instrumental in achieving state-of-th... more While the technique of Deep Neural Networks (DNNs) has been instrumental in achieving state-of-theart results for various Natural Language Processing (NLP) tasks, recent works have shown that the decisions made by DNNs cannot always be trusted. Recently Explainable Artificial Intelligence (XAI) methods have been proposed as a method for increasing DNN's reliability and trustworthiness. These XAI methods are however open to attack and can be manipulated in both white-box (gradient-based) and black-box (perturbationbased) scenarios. Exploring novel techniques to attack and robustify these XAI methods is crucial to fully understand these vulnerabilities. In this work, we propose Tamp-X-a novel attack which tampers the activations of robust NLP classifiers forcing the state-of-the-art white-box and black-box XAI methods to generate misrepresented explanations. To the best of our knowledge, in current NLP literature, we are the first to attack both the white-box and the black-box XAI methods simultaneously. We quantify the reliability of explanations based on three different metrics-the descriptive accuracy, the cosine similarity, and the Lp norms of the explanation vectors. Through extensive experimentation, we show that the explanations generated for the tampered classifiers are not reliable, and significantly disagree with those generated for the untampered classifiers despite that the output decisions of tampered and untampered classifiers are almost always the same. Additionally, we study the adversarial robustness of the tampered NLP classifiers, and find out that the tampered classifiers which are harder to explain for the XAI methods, are also harder to attack by the adversarial attackers.
Disaster analysis in social media content is one of the interesting research domains having abund... more Disaster analysis in social media content is one of the interesting research domains having abundance of data. However, there is a lack of labeled data that can be used to train machine learning models for disaster analysis applications. Active learning is one of the possible solutions to such problem. To this aim, in this paper we propose and assess the efficacy of an active learning based framework for disaster analysis using images shared on social media outlets. Specifically, we analyze the performance of different active learning techniques employing several sampling and disagreement strategies. Moreover, we collect a large-scale dataset covering images from eight common types of natural disasters. The experimental results show that the use of active learning techniques for disaster analysis using images results in a performance comparable to that obtained using human annotated images, and could be used in frameworks for disaster analysis in images without tedious job of manual annotation.
The role of hierarchical entropy analysis in the detection and time-scale determination of covert timing channels
This paper evaluates the potential use of hierarchal entropy analysis to detect covert timing cha... more This paper evaluates the potential use of hierarchal entropy analysis to detect covert timing channels and determine the best time-scale that reveals it. A data transmission simulator is implemented to generate a collection of overt and covert channels. The hierarchical entropy analysis approach is then utilized to detect the covert timing channels and identify the type-scale that provides the highest evidence that the underlying channel is covert. Hierarchical entropy divides the stream of inter-arrival times greedily to identify the time-scale the best reveals the existence of a covert-timing channel. The lowest entropy in the sequence is the best indicator that identifies non-random patterns in the given data stream. The results show that hierarchal entropy analysis performs significantly better than the classical flat entropy approach in the detection of covert timing channels. Furthermore, the hierarchical entropy analysis provides details about the best time-scale that reveals the features of the covert timing channel.
The increasing popularity of social networks and users' tendency towards sharing their feelings, ... more The increasing popularity of social networks and users' tendency towards sharing their feelings, expressions, and opinions in text, visual, and audio content have opened new opportunities and challenges in sentiment analysis. While sentiment analysis of text streams has been widely explored in the literature, sentiment analysis from images and videos is relatively new. This article focuses on visual sentiment analysis in a societally important domain, namely disaster analysis in social media. To this aim, we propose a deep visual sentiment analyzer for disaster-related images, covering different aspects of visual sentiment analysis starting from data collection, annotation, model selection, implementation, and evaluations. For data annotation and analyzing people's sentiments towards natural disasters and associated images in social media, a crowd-sourcing study has been conducted with a large number of participants worldwide. The crowd-sourcing study resulted in a large-scale benchmark dataset with four different sets of annotations, each aiming at a separate task. The presented analysis and the associated dataset, which is made public, will provide a baseline/benchmark for future research in the domain. We believe the proposed system can contribute toward more livable communities by helping different stakeholders, such as news broadcasters, humanitarian organizations, as well as the general public.
2006 IEEE International Conference on Communications, 2006
A mobile Ad-Hoc network (MANET) is a collection of autonomous arbitrarily located wireless mobile... more A mobile Ad-Hoc network (MANET) is a collection of autonomous arbitrarily located wireless mobile hosts, in which an infrastructure is absent. In this paper we propose a fuzzybased hierarchical energy efficient routing scheme (FEER) for large scale mobile ad-hoc networks that aims to maximize the network's lifetime. Each node in the network is characterized by its residual energy, traffic, and mobility. We develop a fuzzy logic controller that combines these parameters, keeping in mind the synergy between them. The value obtained, indicates the importance of a node and it is used in network formation and maintenance. We compare our approach to another energy efficient hierarchical protocol based on the dominating set (DS) idea. Our simulation shows that our design out performs the DS approach in prolonging the network lifetime.
The problem of classifying traffic flows in networks has become more and more important in recent... more The problem of classifying traffic flows in networks has become more and more important in recent times, and much research has been dedicated to it. In recent years, there has been a lot of interest in classifying traffic flows by application, based on the statistical features of each flow. Information about the applications that are being used on a network is very useful in network design, accounting, management, and security. In our previous work we proposed a classification algorithm for Internet traffic flow classification based on Artificial Immune Systems (AIS). We also applied the algorithm on an available data set, and found that the algorithm performed as well as other algorithms, and was insensitive to input parameters, which makes it valuable for embedded systems. It is also very simple to implement, and generalizes well from small training data sets. In this research, we expanded on the previous research by introducing several optimizations in the training and classification phases of the algorithm. We improved the design of the original algorithm in order to make it more predictable. We also give the asymptotic complexity of the optimized algorithm as well as draw a bound on the generalization error of the algorithm. Lastly, we also experimented with several different distance formulas to improve the classification performance. In this paper we have shown how the changes and optimizations applied to the original algorithm do not functionally change the original algorithm, while making its execution 50-60% faster. We also show that the classification accuracy of the Euclidian distance is superseded by the Manhattan distance for this application, giving 1-2% higher accuracy, making the accuracy of the algorithm comparable to that of a Naïve Bayes classifier in previous research that uses the same data set.
In the era of the Internet of Things (IoT), an enormous amount of sensing devices collect and/or ... more In the era of the Internet of Things (IoT), an enormous amount of sensing devices collect and/or generate various sensory data over time for a wide range of fields and applications. Based on the nature of the application, these devices will result in big or fast/real-time data streams. Applying analytics over such data streams to discover new information, predict future insights, and make control decisions is a crucial process that makes IoT a worthy paradigm for businesses and a quality-of-life improving technology. In this paper, we provide a thorough overview on using a class of advanced machine learning techniques, namely Deep Learning (DL), to facilitate the analytics and learning in the IoT domain. We start by articulating IoT data characteristics and identifying two major treatments for IoT data from a machine learning perspective, namely IoT big data analytics and IoT streaming data analytics. We also discuss why DL is a promising approach to achieve the desired analytics in these types of data and applications. The potential of using emerging DL techniques for IoT data analytics are then discussed, and its promises and challenges are introduced. We present a comprehensive background on different DL architectures and algorithms. We also analyze and summarize major reported research attempts that leveraged DL in the IoT domain. The smart IoT devices that have incorporated DL in their intelligence background are also discussed. DL implementation approaches on the fog and cloud centers in support of IoT applications are also surveyed. Finally, we shed light on some challenges and potential directions for future research. At the end of each section, we highlight the lessons learned based on our experiments and review of the recent literature.
In this paper, we present our methods for the MediaEval 2020 Flood Related Multimedia task, which... more In this paper, we present our methods for the MediaEval 2020 Flood Related Multimedia task, which aims to analyze and combine textual and visual content from social media for the detection of realworld flooding events. The task mainly focuses on identifying floods related tweets relevant to a specific area. We propose several schemes to address the challenge. For text-based flood events detection, we use three different methods, relying on Bog of Words (BOW) and an Italian Version of Bert individually and in combination, achieving an F1-score of 0.77%, 0.68%, and 0.70% on the development set, respectively. For the visual analysis, we rely on features extracted via multiple state-of-the-art deep models pre-trained on ImageNet. The extracted features are then used to train multiple individual classifiers whose scores are then combined in a late fusion manner achieving an F1-score of 0.75%. For our mandatory multi-modal run, we combine the classification scores obtained with the best textual and visual schemes in a late fusion manner. Overall, better results are obtained with the multimodal scheme achieving an F1-score of 0.80% on the development set.
In this paper we present our methods for the MediaEval 2019 Multimedia Satellite Task, which is a... more In this paper we present our methods for the MediaEval 2019 Multimedia Satellite Task, which is aiming to extract complementary information associated with adverse events from Social Media and satellites. For the first challenge, we propose a framework jointly utilizing colour, object and scene-level information to predict whether the topic of an article containing an image is a flood event or not. Visual features are combined using early and late fusion techniques achieving an average F1-score of 82.63, 82.40, 81.40 and 76.77. For the multi-modal flood level estimation, we rely on both visual and textual information achieving an average F1-score of 58.48 and 46.03, respectively. Finally, for the flooding detection in time-based satellite image sequences we used a combination of classical computervision and machine learning approaches achieving an average F1score of 58.82%.
The literature shows outstanding capabilities for CNNs in event recognition in images. However, f... more The literature shows outstanding capabilities for CNNs in event recognition in images. However, fewer attempts are made to analyze the potential causes behind the decisions of the models and exploring whether the predictions are based on event-salient objects/regions? To explore this important aspect of event recognition, in this work, we propose an explainable event recognition framework relying on Grad-CAM and an Xception architecture-based CNN model. Experiments are conducted on three large-scale datasets covering a diversified set of natural disasters, social, and sports events. Overall, the model showed outstanding generalization capabilities obtaining overall F1-scores of 0.91, 0.94, and 0.97 on natural disasters, social, and sports events, respectively. Moreover, for subjective analysis of activation maps generated through Grad-CAM for the predicted samples of the model, a crowd-sourcing study is conducted to analyze whether the model's predictions are based on event-related objects/regions or not? The results of the study indicate that 78%, 84%, and 78% of the model decisions on natural disasters, sports, and social events datasets, respectively, are based on event-related objects/regions.
Recent works have shown that deep learning (DL) models can effectively learn city-wide crowd-flow... more Recent works have shown that deep learning (DL) models can effectively learn city-wide crowd-flow patterns, which can be used for more effective urban planning and smart city management. However, DL models have been known to perform poorly on inconspicuous adversarial perturbations. Although many works have studied these adversarial perturbations in general, the adversarial vulnerabilities of deep crowd-flow prediction models in particular have remained largely unexplored. In this paper, we perform a rigorous analysis of the adversarial vulnerabilities of DL-based crowd-flow prediction models under multiple threat settings, making three-fold contributions. (1) We propose CaV-detect by formally identifying two novel properties-Consistency and Validity-of the crowd-flow prediction inputs that enable the detection of standard adversarial inputs with 0% false acceptance rate (FAR). ( ) We leverage universal adversarial perturbations and an adaptive adversarial loss to present adaptive adversarial attacks to evade CaV-detect defense. (3) We propose CVPR, a Consistent, Valid and Physically-Realizable adversarial attack, that explicitly inducts the consistency and validity priors in the perturbation generation mechanism. We find out that although the crowd-flow models are vulnerable to adversarial perturbations, it is extremely challenging to simulate these perturbations in physical settings, notably when CaVdetect is in place. We also show that CVPR attack considerably outperforms the adaptively modified standard attacks in FAR and adversarial loss metrics. We conclude with useful insights emerging from our work and highlight promising future research directions.
Uploads
Papers by Ala Al-Fuqaha