On-line new event detection and tracking

1999

We present results of the University of Iowa topic tracking and detection as well as story segmentation efforts. Topic tracking is performed for the "boundaries given" case. The DET curves for all the runs are consistently smooth and concave suggesting no sudden changes in expectation required from the user. The effect of reducing the training size of relevant stories is examined. The detection runs are performed using a "pipeline" model to utilize the advantage of the deferral period. Performance is strongly influenced by the fact that roughly 2000 to 3000 declared topic clusters are generated during the detection runs. Performance is analyzed with respect to changing the cluster threshold. In segmentation, an agglomerative clustering strategy is adopted. The decision to declare a boundary depends upon both lexical similarity of neighboring segments as well as the pause duration. The algorithmic complexity of the method is O(k log k) where k is the number of pause delimited sentences in the file. The tracking, detection and segmentation modules provide a sound framework for future extension and experimentation.

Proceedings of the 32nd …, 2009

Lecture Notes in Computer Science, 2009

Continuous monitoring of web-based news sources has emerged as a key intelligence task particularly for Homeland Security. We propose a system for web-based news tracking and alerting. Unlike subscription-based alerts, alerting is implemented as a personalized service where the system is trained to recognize potentially important news based on user preferences. Preferences are expressed as combinations of topics and can change dynamically. The system employs Latent Dirichlet Allocation (LDA) for topic discovery and Latent Semantic Indexing (LSI) for alerting.

2004

Abstract New Event Detection is a challenging task that still offers scope for great improvement after years of effort. In this paper we show how performance on New Event Detection (NED) can be improved by the use of text classification techniques as well as by using named entities in a new way. We explore modifications to the document representation in a vector space-based NED system. We also show that addressing named entities preferentially is useful only in certain situations.

Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '05, 2005

Retrospective news event detection (RED) is defined as the discovery of previously unidentified events in historical news corpus. Although both the contents and time information of news articles are helpful to RED, most researches focus on the utilization of the contents of news articles. Few research works have been carried out on finding better usages of time information. In this paper, we do some explorations on both directions based on the following two characteristics of news articles. On the one hand, news articles are always aroused by events; on the other hand, similar articles reporting the same event often redundantly appear on many news sources. The former hints a generative model of news articles, and the latter provides data enriched environments to perform RED. With consideration of these characteristics, we propose a probabilistic model to incorporate both content and time information in a unified framework. This model gives new representations of both news articles and news events. Furthermore, based on this approach, we build an interactive RED system, HISCOVERY, which provides additional functions to present events, Photo Story and Chronicle.

2006

When an important event happens, such as a terrorist attack or natural disaster, many people turn to the World Wide Web to keep track of the most current information. Because large numbers of online agencies report on such events, and continually update their stories, the Web provides timely access to a variety of perspectives. However, following facts in a breaking story is challenging for a number of reasons. For example, news agencies have their own reputation and agenda, such that sources often contradict one another.

Following online news about a specific event can be a difficult task as new information is often scattered across web pages. In such cases, an up-to-date summary of the event would help to inform users and allow them to navigate to articles that are likely to con- tain relevant and novel details. We propose a three-step approach to online news tracking for ad-hoc information needs. First, we con- tinuously cluster the titles of all incoming news articles. Then, we select the clusters that best fit a user’s ad-hoc information need and identify salient sentences. Finally, we select sentences for the sum- mary based on novelty and relevance to the information seen, with- out requiring an a-priori model of events of interest. We evaluate this approach using the 2013 TREC Temporal Summarization test set and show that compared to existing systems our approach re- trieves news facts with significantly higher F-measure and Latency- Discounted Expected Gain. http://dl.acm.org/citation.cfm?id=2809474

1999

In this paper we present two methods for automatic threshold parameter estimation for an event tracking algorithm. We view the threshold as a statistic of the incoming data stream, which is assumed to contain broadcast news stories from radio, television, and newswire sources. Query bias de ned in terms of threshold estimators can be identi ed when a word co-occurrence representation for text is used. Our results suggest that both approaches learn bias from training corpora, leading to improved classi cation accuracy for event tracking applications.

2017 IEEE International Conference on Big Data (Big Data), 2017

In today's era of information overload, people are struggling to detect the evolution of hot topics from massive news media and microblogs such as Twitter. Reports from mainstream news agencies and discussions from microblogs could complement each other to form a complete picture of major events. Existing work has generally focused on a single source, seldom attempting to combine multiple sources to track the evolution of topics: emerging, evolving and fading phrases as this would require a considerably more sophisticated model. This paper proposes a novel story discovery model that integrates evolutionary topics in news and Twitter data sources using an incremental algorithm by 1) discovering complementary information from news and microblogs that provides a more complete view of major events; 2) modeling emerging, evolving and fading topics and features throughout ongoing events; and 3) creating a scalable algorithm that is capable of handling massive data from news and social...

The Information Retrieval Series, 2002

We present an approach to detection and tracking of topics in multilingual broadcast news based upon a dynamic clustering scheme. Our approach derives from a system used to filter Web searches from multiple sources, with extensions for pipelining document clusters, part-of-speech tagging and extraction of named entities for use in an extended similarity measure.