Event detection in Text Streams

Explainability in the field of event detection is a new emerging research area. For practitioners and users alike, explainability is essential to ensuring that models are widely adopted and trusted. Several research efforts have focused on the efficacy and efficiency of event detection. However, a human-centric explanation approach to existing event detection solutions is still
lacking. This paper presents an overview of a conceptual framework for human-centric semantic-based explainable event detection with the acronym HUSEED. The framework considered the affordances of XAI and semantics technologies for human-comprehensible explanations of events to facilitate 5W1H explanations (Who did what, when, where, why, and how). Providing
this kind of explanation will lead to trustworthy, unambiguous, and transparent event detection models with a higher possibility of uptake by users in various domains of application. We illustrated the applicability of the proposed framework by using two use cases involving first story detection and fake news detection.

Cybersecurity event detection is a crucial problem for mitigating effects on various aspects of society. Social media has become a notable source of indicators for detection of diverse events. Though previous social media based strategies for cybersecurity event detection focus on mining certain event-related words, the dynamic and evolving nature of online discourse limits the performance of these approaches. Further, because these are typically unsupervised or weakly supervised learning strategies, they do not perform well in an environment of biased samples, noisy context, and informal language which is routine for online, user-generated content. This paper takes a supervised learning approach by proposing a novel multi-task learning based model. Our model can handle diverse structures in feature space by learning models for different types of potential high-profile targets simultaneously. For parameter optimization, we develop an efficient algorithm based on the alternating direction method of multipliers. Through extensive experiments on a real world Twitter dataset, we demonstrate that our approach consistently outperforms existing methods at encoding and identifying cybersecurity incidents.

Social media data streams are an invaluable source for timely and up-to-date information about current events. As a consequence, several event detection techniques have been proposed in the literature in order to tap this information source. However, most of these proposals focus on the information extraction aspect of the problem and tend to ignore the streaming nature of the input. The work conducted in our research group therefore intends to address these stream-related challenges, such as detecting events incrementally, reporting them in (near) real-time, and coping with fluctuations and spikes in the social media data stream. In this article, we report on the results that we obtained so far and outline our research agenda for the remainder of this work.

Twitter's popularity as a source of up-to-date news and information is constantly increasing. In response to this trend, numerous event detection techniques have been proposed to cope with the rate and volume of Twitter data streams. Although most of these works conduct some evaluation of the proposed technique, a comparative study is often omitted. In this paper, we present a survey and experimental analysis of state-of-the-art event detection techniques for Twitter data streams. In order to conduct this study, we define a series of measures to support the quantitative and qualitative comparison. We demonstrate the effectiveness of these measures by applying them to event detection techniques as well as to baseline approaches using real-world Twitter streaming data.

First Story Detection is hard because the most accurate systems become progressively slower with each document processed. We present a novel approach to FSD, which operates in constant time/space and scales to very high volume streams. We show that when computing novelty over a large dataset of tweets, our method performs 192 times faster than a state-of-the-art baseline without sacrificing accuracy. Our method is capable of performing FSD on the full Twitter stream on a single core of modest hardware.

Over the last decade, the infrastructure supporting the smart city has lived together with and was surpassed by the rise of social media. The tremendous growth of both mobile devices and social media users has unearthed a new kind of services in the so-called location-based social networks (LBSNs). In this new scenario, the term crowdsensing refers to sharing data collected by sensing humans with the aim of measuring phenomena of common interest. Crowd-sourced location data provide the ability to study, for the first time, the movement of individuals in urban environments. In this paper, we address the problem of monitoring crowds, whereabouts and movement, which can assist decision making in education, emergency training, urban planning, traffic engineering, etc. Precisely, two-phase density-based analysis for collectives and crowds (2PD-CC) is a novel methodology over public data in LBSN, which combines density-based clustering, outlier detection a topic modeling over a region under study to detect, predict, and explain abnormal group behavior. In order to validate the methodology and its potential application to full-scale problems, an experiment over Twitter data was performed in Madrid city.

Information about events happening in the real world are generated online on social media in real-time. There is substantial research done to detect these events using information posted on websites like Twitter, Tumblr, and Instagram. The information posted depends on the type of platform the website relies upon, such as short messages, pictures, and long form articles. In this paper, we extend an existing real-time event detection at onset approach to include multiple websites. We present three different approaches to merging information from two different social media sources. We also analyze the strengths and weaknesses of these approaches. We validate the detected events using newswire data that is collected during the same time period. Our results show that including multiple sources increases the number of detected events and also increase the quality of detected events.

Information about events happening in the real world are generated online on social media in real-time. There is substantial research done to detect these events using information posted on websites like Twitter, Tumblr, and Instagram. The information posted depends on the type of platform the website relies upon, such as short messages, pictures, and long form articles. In this paper, we extend an existing real-time event detection at onset approach to include multiple websites. We present three different approaches to merging information from two different social media sources. We also analyze the strengths and weaknesses of these approaches. We validate the detected events using newswire data that is collected during the same time period. Our results show that including multiple sources increases the number of detected events and also increase the quality of detected events.

Web mining is the application of data mining techniques to discover patterns from the Web. Topic tracking is one of the technologies that has been developed and can be used in the text mining process. The main purpose of topic tracking is to identify and follow events presented in multiple news sources, including newswires, radio and TV broadcasts. In this paper, a survey of topic tracking techniques is presented. KeywordsText Mining, Topic detection, topic tracking

In recent year, advances in hardware and information technology have led to large flow of data across over IP networks. Continuous burst detection in data streams has important applications in fraud credit card detection, real-time IP traffic analysis, sensor readings, email/SMS/blog and hot trends detection in social text sources. The arrival and departure of data objects in a streaming manner impose new challenges for burst detection algorithms, especially in time and space efficiency. Moreover, sample time window size and burst defined threshold values are important to detect accurate burst in continuous data stream. Thus in this work, we describe how to adjust time windows sizes and burst defined threshold values for burst trends detection in real time. And then, correlated bursty terms are detected by finding correlations among them. The burst detection is the task of finding unexpected change in some quantity in real time tweet stream. The main objective of this work is timely detection of bursty trends in Twitter tweet stream which have happened recently and discovery of their evolutionary patterns along the timeline. Our experimental results show the scalability, applicability and effectiveness of our approach in real time processing.

Analysis of public behavior plays an important role in crisis management, disaster response, and evacuation planning. Unfortunately, collecting relevant data can be costly and finding meaningful information for analysis is challenging. A growing number of Location-based Social Network services provides time-stamped, geo-located data that opens new opportunities and solutions to a wide range of challenges. Such spatiotemporal data has substantial potential to increase situational awareness of local events and improve both planning and investigation. However, the large volume of unstructured social media data hinders exploration and examination. To analyze such social media data, our system provides the analysts with an interactive visual spatiotemporal analysis and spatial decision support environment that assists in evacuation planning and disaster management. We demonstrate how to improve investigation by analyzing the extracted public behavior responses from social media before, during and after natural disasters, such as hurricanes and tornadoes.

Social media data (SMD) is driven by statistical and analytical technologies to obtain information for various decisions. SMD is vast and evolutionary in nature which makes traditional data warehouses ill suited. The research aims to propose and implement novel framework that analyze tweets data from online social networking site (OSN; i.e., Twitter). The authors fetch streaming tweets from Twitter API using Apache Flume to detect clusters of users having similar sentiment. Proposed approach utilizes scalable and fault tolerant system (i.e., Hadoop) that typically harness HDFS for data storage and map-reduce paradigm for data processing. Apache Hive is used to work on top of Hadoop for querying data. The experiments are performed to test the scalability of proposed framework by examining various sizes of data. The authors' goal is to handle big social data effectively using cost-effective tools for fetching as well as querying unstructured data and algorithms for analysing scala...

From last few decades there is wide spread usage of social network platforms such as twitter or other micro blogging systems which contains huge amount of timely generated data. Tweeter is fastest means of information sharing where user shares event/news which take place in front of them. Thus Tweeter act as news portal where news reaches to the people within fraction of seconds. Extracting valuable information in timely manner is important because this wealthy information is useful for companies, government agencies and health organizations. Topic detection is the new research area in data mining and knowledge discovery where extracting useful and valuable information from timely generated online streams is the new challenge. In this article we survey the different algorithms used for trending topic and event detection using social media data and proposes new system for topic detection from social media.

In recent year, advances in hardware and information technology have led to large flow of data across over IP networks. Continuous burst detection in data streams has important applications in fraud credit card detection, real-time IP traffic analysis, sensor readings, email/SMS/blog and hot trends detection in social text sources. The arrival and departure of data objects in a streaming manner impose new challenges for burst detection algorithms, especially in time and space efficiency. Moreover, sample time window size and burst defined threshold values are important to detect accurate burst in continuous data stream. Thus in this work, we describe how to adjust time windows sizes and burst defined threshold values for burst trends detection in real time. And then, correlated bursty terms are detected by finding correlations among them. The burst detection is the task of finding unexpected change in some quantity in real time tweet stream. The main objective of this work is timely detection of bursty trends in Twitter tweet stream which have happened recently and discovery of their evolutionary patterns along the timeline. Our experimental results show the scalability, applicability and effectiveness of our approach in real time processing.

Social media websites can be used as a data source for mining public opinion on a variety of subjects including climate change. Twitter, in particular, allows for the evaluation of public opinion across both time and space because geotagged tweets include timestamps and geographic coordinates (latitude/longitude). In this study, a large dataset of geotagged tweets containing certain keywords relating to climate change is analyzed using volume analysis and text mining techniques such as topic modeling and sentiment analysis. Latent Dirichlet allocation was applied for topic modeling to infer the different topics of discussion, and Valence Aware Dictionary and sEntiment Reasoner was applied for sentiment analysis to determine the overall feelings and attitudes found in the dataset. These techniques are used to compare and contrast the nature of climate change discussion between different countries and over time. Sentiment analysis shows that the overall discussion is negative, especially when users are reacting to political or extreme weather events. Topic modeling shows that the different topics of discussion on climate change are diverse, but some topics are more prevalent than others. In particular, the discussion of climate change in the USA is less focused on policy-related topics than other countries.

Standalone systems cannot handle the giant traffic loads generated by Twitter due to memory constraints. A parallel computational environment provided by Apache Hadoop can distribute and process the data over different destination systems. In this paper, the Hadoop cluster with four nodes integrated with RHadoop, Flume, and Hive is created to analyze the tweets gathered from the Twitter stream. Twitter stream data is collected relevant to an event/topic like IPL-2015, cricket, Royal Challengers Bangalore, Kohli, Modi, from May 24 to 30, 2016 using Flume. Hive is used as a data warehouse to store the streamed tweets. Twitter analytics like maximum number of tweets by users, the average number of followers, and maximum number of friends are obtained using Hive. The network graph is constructed with the user's unique screen name and mentions using 'R'. A timeline graph of individual users is generated using 'R'. Also, the proposed solution analyses the emotions of cricket fans by classifying their Twitter messages into appropriate emotional categories using the optimized support vector neural network (OSVNN) classification model. To attain better classification accuracy, the performance of SVNN is enhanced using a chimp optimization algorithm (ChOA). Extracting the users' emotions toward an event is beneficial for prediction, but when coupled with visualizations, it becomes more powerful. Bar-chart and wordcloud are generated to visualize the emotional analysis results.

Content shared in microblogs during disasters is expressed in various formats and languages. This diversity makes the information retrieval process more complex and computationally infeasible in real time. To address this, we propose a classification model for the identification of prominent users who are sharing relevant and exclusive information during the disaster. Users who have shared at least one tweet about the disaster are modeled using three kinds of time-sensitive features, including topical, social and geographical features. Then, these users are classified into two classes using a linear Support Vector Machine (SVM) to evaluate them over the extracted features and identify the most prominent ones. The first results using the actual dataset, show that our model has a high accuracy by detecting most of the prominent users. Moreover, we demonstrate that all the proposed features used by our model are indispensable to achieve this high accuracy.

User-generated content in social media can offer valuable insights into local trends, events, and topics of interest. However, navigating through the vast amounts of posts either to retrieve certain pieces of information or to obtain an overview of the existing content, is often a challenging and overwhelming task. In this work, we present μTOP, a system for detecting and summarizing locally trending topics in microblog posts based on spatial, temporal and textual criteria. Using a sliding window model over an incoming stream of posts, μTOP detects locally trending topics, and associates each one with a spatio-temporal footprint. Then, for each spatial region and time period in which a certain topic is trending, the system generates a summary of the relevant posts, by selecting top-k posts based on the criteria of coverage and diversity. μTOP includes a Web-based user interface, providing a comprehensive way to visualize and explore the detected topics and their spatio-temporal summ...

Standalone systems cannot handle the giant traffic loads generated by Twitter due to memory constraints. A parallel computational environment provided by Apache Hadoop can distribute and process the data over different destination systems. In this paper, the Hadoop cluster with four nodes integrated with RHadoop, Flume, and Hive is created to analyze the tweets gathered from the Twitter stream. Twitter stream data is collected relevant to an event/topic like IPL-2015, cricket, Royal Challengers Bangalore, Kohli, Modi, from May 24 to 30, 2016 using Flume. Hive is used as a data warehouse to store the streamed tweets. Twitter analytics like maximum number of tweets by users, the average number of followers, and maximum number of friends are obtained using Hive. The network graph is constructed with the user's unique screen name and mentions using 'R'. A timeline graph of individual users is generated using 'R'. Also, the proposed solution analyses the emotions of cricket fans by classifying their Twitter messages into appropriate emotional categories using the optimized support vector neural network (OSVNN) classification model. To attain better classification accuracy, the performance of SVNN is enhanced using a chimp optimization algorithm (ChOA). Extracting the users' emotions toward an event is beneficial for prediction, but when coupled with visualizations, it becomes more powerful. Bar-chart and wordcloud are generated to visualize the emotional analysis results.

Twitter's increasing popularity as a source of up to date news and information about current events has spawned a body of research on event detection techniques for social media data streams. Although all proposed approaches provide some evidence as to the quality of the detected events, none relate this task based performance to their run time performance in terms of processing speed, data throughput, or memory usage. In parti cular, neither a quantitative nor a comparative evaluation of these aspects has been performed to date. In this article, we study the run time and task based performance of several state of the art event detection techniques for Twitter. In order to reproducibly compare run time performance, our approach is based on a general purpose data stream management system, whereas task based performance is automatically assessed based on a series of novel measures.

Twitter's increasing popularity as a source of up to date news and information about current events has spawned a body of research on event detection techniques for social media data streams. Although all proposed approaches provide some evidence as to the quality of the detected events, none relate this task-based performance to their runtime performance in terms of processing speed or data throughput. In particular, neither a quantitative nor a comparative evaluation of these aspects has been performed to date. In this paper, we study the runtime and task-based performance of several state-of-the-art event detection techniques for Twitter. In order to reproducibly compare run-time performance, our approach is based on a general-purpose data stream management system, whereas task-based performance is automatically assessed based on a series of novel measures.

Nowadays, there are plenty of sources generating massive amounts of text data streams in a continuous way. For example, the increasing popularity and the active use of social networks result in voluminous and fastflowing text data streams containing a large amount of user-generated data about almost any topic around the world. However, the observation and tracking of the ongoing evolution of topics in these unevenly distributed text data streams is a challenging task for analysts, news reporters, or other users. This paper presents "Store-Motion" a shape-based visualization to track the ongoing evolution of topics' frequency (i.e., importance), sentiment (i.e., emotion), and context (i.e., story) in user-defined topic channels over continuous flowing text data streams. The visualization supports the user in keeping the overview over vast amounts of streaming data and guides the perception of the user to unexpected and interesting points or periods in the text data stream. In this work, we mainly focus on the visualization of text streams from the social microblogging service Twitter, for which we present a series of case studies (e.g., the observation of cities, movies, or natural disasters) applied on real-world data streams collected from the public timeline. However, to further evaluate our visualization, we also present a baseline case study applied on the text stream of a fantasy book series.

Social media websites can be used as a data source for mining public opinion on a variety of subjects including climate change. Twitter, in particular, allows for the evaluation of public opinion across both time and space because geotagged tweets include timestamps and geographic coordinates (latitude/longitude). In this study, a large dataset of geotagged tweets containing certain keywords relating to climate change is analyzed using volume analysis and text mining techniques such as topic modeling and sentiment analysis. Latent Dirichlet allocation was applied for topic modeling to infer the different topics of discussion, and Valence Aware Dictionary and sEntiment Reasoner was applied for sentiment analysis to determine the overall feelings and attitudes found in the dataset. These techniques are used to compare and contrast the nature of climate change discussion between different countries and over time. Sentiment analysis shows that the overall discussion is negative, especially when users are reacting to political or extreme weather events. Topic modeling shows that the different topics of discussion on climate change are diverse, but some topics are more prevalent than others. In particular, the discussion of climate change in the USA is less focused on policy-related topics than other countries.

Content shared in microblogs during disasters is expressed in various formats and languages. This diversity makes the information retrieval process more complex and computationally infeasible in real time. To address this, we propose a classification model for the identification of prominent users who are sharing relevant and exclusive information during the disaster. Users who have shared at least one tweet about the disaster are modeled using three kinds of time-sensitive features, including topical, social and geographical features. Then, these users are classified into two classes using a linear Support Vector Machine (SVM) to evaluate them over the extracted features and identify the most prominent ones. The first results using the actual dataset, show that our model has a high accuracy by detecting most of the prominent users. Moreover, we demonstrate that all the proposed features used by our model are indispensable to achieve this high accuracy.

In recent year, advances in hardware and information technology have led to large flow of data across over IP networks. Continuous burst detection in data streams has important applications in fraud credit card detection, real-time IP traffic analysis, sensor readings, email/SMS/blog and hot trends detection in social text sources. The arrival and departure of data objects in a streaming manner impose new challenges for burst detection algorithms, especially in time and space efficiency. Moreover, sample time window size and burst defined threshold values are important to detect accurate burst in continuous data stream. Thus in this work, we describe how to adjust time windows sizes and burst defined threshold values for burst trends detection in real time. And then, correlated bursty terms are detected by finding correlations among them. The burst detection is the task of finding unexpected change in some quantity in real time tweet stream. The main objective of this work is timely detection of bursty trends in Twitter tweet stream which have happened recently and discovery of their evolutionary patterns along the timeline. Our experimental results show the scalability, applicability and effectiveness of our approach in real time processing.

Detection of events using voluntarily generated content in microblogs has been the objective of numerous recent studies. One essential challenge tackled in these studies is estimating the locations of events. In this paper, we review the state-of-the-art location estimation techniques used in the localization of events detected in microblogs, particularly in Twitter, which is one of the most popular microblogging platforms worldwide. We analyze these techniques with respect to the targeted event type, granularity of estimated locations, location-related features selected as sources of spatial evidence, and the method used to make aggregate decisions based on the extracted evidence. We discuss the strengths and advantages of alternative solutions to various problems related to location estimation, as well as their preconditions and limitations. We examine the most widely used evaluation methods to analyze the accuracy of estimations and present the results reported in the literature. We also discuss our findings and highlight important research challenges that may need further attention.

Monitoring and analysis of streaming data, such as social media, sensors, and news feeds, has become increasingly important for business and government. Volume and velocity of incoming data are key challenges. To effectively support monitoring and analysis, statistical and visual analytics techniques need to be seamlessly integrated; analytic techniques for a variety of data types (e.g. text, numerical) and scope (e.g. incremental, rolling-window, and global) must be properly accommodated; interaction and coordination among several visualizations must be supported in an efficient manner; and the system should support the use of different analytics techniques in a pluggable and collaborative manner. Especially in web based environments, these requirements pose restrictions on the basic architecture for such systems. In this paper we report on our experience of building a reference web architecture for real-time visual analytics of streaming data, identify and discuss architectural patterns that address these challenges, and report on applying the reference architecture for real-time Twitter monitoring.

While the salience of social media platforms on modern interactive communication between diverse social actors has been demonstrated, less academic attention has been paid to comparisons between framed topics and user interactions across social media platforms, such as Twitter and Weibo. This article suggests text mining and natural language processing tools for cross-platform comparative social media studies, based on Latent Dirichlet Allocation (LDA) and social network analysis. This study illustrates how the suggested topic models and data processing algorithms can be applied to a real-life example (U.S.-China trade war discourse on social media), and experimented the methods on social media text mining data, revealing differences between user interactions on Twitter, predominantly "Western," and Weibo, largely representing Chinesespeaking users. We discuss the strengths and weaknesses of the suggested machine learning algorithms for comparative social media studies.

While the salience of social media platforms on modern interactive communication between diverse social actors has been demonstrated, less academic attention has been paid to comparisons between framed topics and user interactions across social media platforms, such as Twitter and Weibo. This article suggests text mining and natural language processing tools for cross-platform comparative social media studies, based on Latent Dirichlet Allocation (LDA) and social network analysis. This study illustrates how the suggested topic models and data processing algorithms can be applied to a real-life example (U.S.-China trade war discourse on social media), and experimented the methods on social media text mining data, revealing differences between user interactions on Twitter, predominantly "Western," and Weibo, largely representing Chinesespeaking users. We discuss the strengths and weaknesses of the suggested machine learning algorithms for comparative social media studies.

This article explores the ICT infrastructure enabling social media usage for disaster management. In the era of pervasive social media, the study seeks to probe the usage of social media tools and functionalities in times of disasters. The study deploys Grounded Theory approach. Extending on disaster management cycle, this study examines the various source of information in social media followed by the type of information disseminated by them and additionally the most extensively used form of information. Three recent catastrophic events in disaster-prone Asia Pacific region were selected, Kashmir floods (September 2014), Indonesia landslide (December 2014), and Cyclone Pam (March 2015). Twitter was deployed for data collection under two phases, immediately aftermath of the event and a random sample from the first 5 days of the event. These two data sets examined the research questions by presenting timely and panoramic dimensions. The study is novel since it extends the literature ...

Content shared in microblogs during disasters is expressed in various formats and languages. This diversity makes the information retrieval process more complex and computationally infeasible in real time. To address this, we propose a classification model for the identification of prominent users who are sharing relevant and exclusive information during the disaster. Users who have shared at least one tweet about the disaster are modeled using three kinds of time-sensitive features, including topical, social and geographical features. Then, these users are classified into two classes using a linear Support Vector Machine (SVM) to evaluate them over the extracted features and identify the most prominent ones. The first results using the actual dataset, show that our model has a high accuracy by detecting most of the prominent users. Moreover, we demonstrate that all the proposed features used by our model are indispensable to achieve this high accuracy.

Social networking sites such as Flickr, YouTube, Facebook, etc. contain a huge amount of user-contributed data for a variety of real-world events. These events can be some natural calamities such as earthquakes, floods, forest fires, etc. or some man-made hazards like riots. This work focuses on getting better knowledge about a natural hazard event using the data available from social networking sites. Rescue and relief activities in emergency situations can be enhanced by identifying sub-events of a particular event. Traditional topic discovery techniques used for event identification in news data cannot be used for social media data because social network data may be unstructured. To address this problem the features or metadata associated with social media data can be exploited. These features can be user-provided annotations (e.g., title, description) and automatically generated information (e.g., content creation time). Considerable improvement in performance is observed by using multiple features of social media data for sub-event detection rather than using individual feature. Proposed here is a two-step process. In the first step, clusters are formed from social network data using relevant features individually. Based on the significance of features weights are assigned to them. And in the second step all the clustering solutions formed in the first step are combined in a principal weighted manner to give the final clustering solution. Each cluster represents a sub-event for a particular natural hazard.

Social networking sites such as Flickr, YouTube, Facebook, etc. contain a huge amount of user-contributed data for a variety of real-world events. These events can be some natural calamities such as earthquakes, floods, forest fires, etc. or some man-made hazards like riots. This work focuses on getting better knowledge about a natural hazard event using the data available from social networking sites. Rescue and relief activities in emergency situations can be enhanced by identifying sub-events of a particular event. Traditional topic discovery techniques used for event identification in news data cannot be used for social media data because social network data may be unstructured. To address this problem the features or metadata associated with social media data can be exploited. These features can be user-provided annotations (e.g., title, description) and automatically generated information (e.g., content creation time). Considerable improvement in performance is observed by using multiple features of social media data for sub-event detection rather than using individual feature. Proposed here is a two-step process. In the first step, clusters are formed from social network data using relevant features individually. Based on the significance of features weights are assigned to them. And in the second step all the clustering solutions formed in the first step are combined in a principal weighted manner to give the final clustering solution. Each cluster represents a sub-event for a particular natural hazard.

Event Detection has been one of the research areas in Text Mining that has attracted attention during this decade due to the widespread availability of social media data specifically twitter data. Twitter has become a major source for information about real-world events because of the use of hashtags and the small word limit of Twitter that ensures concise presentation of events. Previous works on event detection from tweets are either applicable to detect localized events or breaking news only or miss out on many important events. This paper presents the problems associated with event detection from tweets and a tweet-segmentation based system for event detection called SEDTWik, an extension to a previous work, that is able to detect newsworthy events occurring at different locations of the world from a wide range of categories. The main idea is to split each tweet and hash-tag into segments, extract bursty segments, cluster them, and summarize them. We evaluated our results on the...

This is an era where people are too busy to check out their inbox. They swipe off notifications, not knowing they might miss something important. These could be anything like, personal messages asking whether you are free to meet up or reminders to events that you need to attend or some emails regarding interesting events that are happening around you. The need of the hour is an efficient way for keeping track of the important events from the vast number of incoming messages. This paper proposes a solution based on event extraction from emails. The scope of the project is limited to Gmail. The primary component is an android application that identifies event containing emails, extracts important details and provides an automated reminder system for the same depending on user needs. Various machine learning techniques along with natural language processing are used for the fulfillment for this project.

User communities in social networks are usually identified by considering explicit structural social connections between users. While such communities can reveal important information about their members such as family or friendship ties and geographical proximity, they do not necessarily succeed at pulling like-minded users that share the same interests together. In this paper, we are interested in identifying communities of users that share similar topical interests over time, regardless of whether they are explicitly connected to each other on the social network. More specifically, we tackle the problem of identifying temporal topic-based communities from Twitter, i.e., communities of users who have similar temporal inclination towards the current emerging topics on Twitter. We model each topic as a collection of highly correlated semantic concepts observed in tweets and identify them by clustering the time-series based representation of each concept built based on each concept&#...

Event Detection has been one of the research areas in Text Mining that has attracted attention during this decade due to the widespread availability of social media data specifically twitter data. Twitter has become a major source for information about real-world events because of the use of hashtags and the small word limit of Twitter that ensures concise presentation of events. Previous works on event detection from tweets are either applicable to detect localized events or breaking news only or miss out on many important events. This paper presents the problems associated with event detection from tweets and a tweet-segmentation based system for event detection called SEDTWik, an extension to a previous work, that is able to detect newsworthy events occurring at different locations of the world from a wide range of categories. The main idea is to split each tweet and hash-tag into segments, extract bursty segments, cluster them, and summarize them. We evaluated our results on the...

Twitter provides the freshest source of data about what is happening in the lives people across the world. The publicly available streams of status updates available on Twitter have been used to track earthquakes, forest fires and most especially flu outbreaks. Current techniques for tracking flu outbreaks rely on count data for a number of keywords. However, count data alone on the noisy Twitter streams is not reliable enough for health officials to make critical decisions. We propose a semi-automatic outbreak detection system. Rather than providing only alarms backed by count data, we propose a summarization system that will allow health officials to quickly verify outbreak alarms. This will lead to higher levels of trust in the system and allow the system to be used by health organizations around the world. We experimentally verify our summarization system and have found system users to have an accuracy of 0.86 when identifying multitweet summaries.

In today’s world the internet and social media are omnipresent and information is accessible to everyone. This shifted the advantage from those who have access to information to those who do so first. Identifying new events as they emerge is of substantial value to financial institutions who consider realtime information in their decision making processes, as well as for journalists that report about breaking news and governmental agencies that collect information and respond to emergencies. First Story Detection is the task of identifying those documents in a stream of documents that talk about new events first. This seemingly simple task is non-trivial as the computational effort increases with every processed document. Standard approaches to solve First Story Detection determine a document’s novelty by comparing it to previously seen documents. This results in the highest reported accuracy but even the currently fastest system only scales to 10% of the Twitter stream. In this the...

Social media data (SMD) is driven by statistical and analytical technologies to obtain information for various decisions. SMD is vast and evolutionary in nature which makes traditional data warehouses ill suited. The research aims to propose and implement novel framework that analyze tweets data from online social networking site (OSN; i.e., Twitter). The authors fetch streaming tweets from Twitter API using Apache Flume to detect clusters of users having similar sentiment. Proposed approach utilizes scalable and fault tolerant system (i.e., Hadoop) that typically harness HDFS for data storage and map-reduce paradigm for data processing. Apache Hive is used to work on top of Hadoop for querying data. The experiments are performed to test the scalability of proposed framework by examining various sizes of data. The authors' goal is to handle big social data effectively using cost-effective tools for fetching as well as querying unstructured data and algorithms for analysing scala...

We introduce our methodology for collecting tweets and identifying event-related actionable information using key terms and inflections. The vast amount of user-generated content makes it challenging to detect relevant information. Therefore, we aim to facilitate extracting morphological, syntactic, and semantic features of a key term semiautomatically. The results of our study show that handling the inflections of key terms separately has its advantages: in the disaster scenario that we are investigating we are successful in discovering relevant and irrelevant information effectively.

People use Online Social Networks (OSNs) to express their opinions and feelings about many topics. Depending on the nature of an event and its dissemination rate in OSNs, and considering specific regions, the users' behavior can drastically change over a specific period of time. In this context, this work aims to propose an event detection system at the early stages of an event based on changes in the users' behavior in an OSN. This system can detect an event of any subject, and thus, it can be used for different purposes. The proposed event detection system is composed of the following main modules: (1) determination of the user's location, (2) message extraction from an OSN, (3) topic identification using natural language processing (NLP) based on the Deep Belief Network (DBN), (4) the user behavior change analyzer in the OSN, and (5) affective analysis for emotion identification based on a tree-convolutional neural network (tree-CNN). In the case of public health, the early event detection is very relevant for the population and the authorities in order to be able take corrective actions. Hence, the new coronavirus disease (COVID-19) is used as a case study in this work. For performance validation, the modules related to the topic identification and affective analysis were compared with other similar solutions or implemented with other machine learning algorithms. In the performance assessment, the proposed event detection system achieved an accuracy higher than 0.90, while other similar methods reached accuracy values less than 0.74. Additionally, our proposed system was able to detect an event almost three days earlier than the other methods. Furthermore, the information provided by the system permits to understand the predominant characteristics of an event, such as keywords and emotion type of messages. INDEX TERMS Event detection, online social networks, affective analysis, natural language processing, COVID-19.

People use Online Social Networks (OSNs) to express their opinions and feelings about many topics. Depending on the nature of an event and its dissemination rate in OSNs, and considering specific regions, the users' behavior can drastically change over a specific period of time. In this context, this work aims to propose an event detection system at the early stages of an event based on changes in the users' behavior in an OSN. This system can detect an event of any subject, and thus, it can be used for different purposes. The proposed event detection system is composed of the following main modules: (1) determination of the user's location, (2) message extraction from an OSN, (3) topic identification using natural language processing (NLP) based on the Deep Belief Network (DBN), (4) the user behavior change analyzer in the OSN, and (5) affective analysis for emotion identification based on a tree-convolutional neural network (tree-CNN). In the case of public health, the early event detection is very relevant for the population and the authorities in order to be able take corrective actions. Hence, the new coronavirus disease (COVID-19) is used as a case study in this work. For performance validation, the modules related to the topic identification and affective analysis were compared with other similar solutions or implemented with other machine learning algorithms. In the performance assessment, the proposed event detection system achieved an accuracy higher than 0.90, while other similar methods reached accuracy values less than 0.74. Additionally, our proposed system was able to detect an event almost three days earlier than the other methods. Furthermore, the information provided by the system permits to understand the predominant characteristics of an event, such as keywords and emotion type of messages. INDEX TERMS Event detection, online social networks, affective analysis, natural language processing, COVID-19.

Social media data (SMD) is driven by statistical and analytical technologies to obtain information for various decisions. SMD is vast and evolutionary in nature which makes traditional data warehouses ill suited. The research aims to propose and implement novel framework that analyze tweets data from online social networking site (OSN; i.e., Twitter). The authors fetch streaming tweets from Twitter API using Apache Flume to detect clusters of users having similar sentiment. Proposed approach utilizes scalable and fault tolerant system (i.e., Hadoop) that typically harness HDFS for data storage and map-reduce paradigm for data processing. Apache Hive is used to work on top of Hadoop for querying data. The experiments are performed to test the scalability of proposed framework by examining various sizes of data. The authors' goal is to handle big social data effectively using cost-effective tools for fetching as well as querying unstructured data and algorithms for analysing scala...

Social media data (SMD) is driven by statistical and analytical technologies to obtain information for various decisions. SMD is vast and evolutionary in nature which makes traditional data warehouses ill suited. The research aims to propose and implement novel framework that analyze tweets data from online social networking site (OSN; i.e., Twitter). The authors fetch streaming tweets from Twitter API using Apache Flume to detect clusters of users having similar sentiment. Proposed approach utilizes scalable and fault tolerant system (i.e., Hadoop) that typically harness HDFS for data storage and map-reduce paradigm for data processing. Apache Hive is used to work on top of Hadoop for querying data. The experiments are performed to test the scalability of proposed framework by examining various sizes of data. The authors' goal is to handle big social data effectively using cost-effective tools for fetching as well as querying unstructured data and algorithms for analysing scala...

Public infrastructure systems provide many of the services that are critical to the health, functioning, and security of society. Many of these infrastructures, however, lack continuous physical sensor monitoring to be able to detect failure events or damage that has occurred to these systems. We propose the use of social sensor big data to detect these events. We focus on two main infrastructure systems, transportation and energy, and use data from Twitter streams to detect damage to bridges, highways, gas lines, and power infrastructure. Through a three-step filtering approach and assignment to geographical cells, we are able to filter out noise in this data to produce relevant geolocated tweets identifying failure events. Applying the strategy to real-world data, we demonstrate the ability of our approach to utilize social sensor big data to detect damage and failure events in these critical public infrastructures.

The Social Internet of Things (SIoT) paradigm incorporates social networking concepts with the Internet of Things (IoT) solutions to support novel services. The massive amount of data (big data) produced by SIoT necessitates efficient information processing frameworks to exploit social relationships and comprehend actionable information from real-world observations. Data from AIenabled sensors (AIS) is typically geo-tagged, thus demanding geotextual processing for information retrieval and analysis. Social media applications are the main source of geo-textual data as mobile users connect with millions of posts daily. The processing of big geo-textual data requires resource-efficient algorithms and frameworks. Clustering algorithms are often applied to geo-textual data to examine spatial, textual, and temporal information for event detection, sentiment analysis, and search query response. Clustering algorithms on big data are resource-hungry requiring comparisons among all data points to calculate similarity and distance metrics. Existing hybrid clustering techniques execute algorithms collectively on geotextual data resulting in a enormous footprint for big data. We propose a resource-efficient clustering framework for AIS that hierarchically performs geo-textual clustering without significantly lowering the clustering quality. The proposed framework achieves substantial time and memory efficiency while reducing the overall resource requirements for constrained end-user and edge devices compared to the standard hybrid geo-textual clustering framework. Moreover, we augment the research work by developing open-source scripts for both hierarchical and hybrid clustering frameworks.