Social Media Analytics for Crisis Response
2016
Abstract
AI
AI
Crises or large-scale emergencies such as earthquakes and hurricanes cause massive damage to lives and property. Crisis response is an essential task to mitigate the impact of a crisis. An effective response to a crisis necessitates information gathering and analysis. Traditionally, this process has been restricted to the information collected by first responders on the ground in the affected region or by official agencies such as local governments involved in the response. However, the ubiquity of mobile devices has empowered people to publish information during a crisis through social media, such as the damage reports from a hurricane. Social media has thus emerged as an important channel of information which can be leveraged to improve crisis response.
References (117)
- 2 Search Interface in TweetTracker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
- 3 Visual Analytics in TweetXplorer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
- 5 Retweet Network Identifying the Direction and Involved Nodes in In- formation Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
- 6 Parameters Used to Collect Tweets Related to Hurricane Sandy . . . . . . . 13
- Pre-Landfall Discussion of Hurricane Sandy . . . . . . . . . . . . . . . . . . . . . . . . . . 15
- 9 Discussion of Hurricane Sandy Immediately After Landfall . . . . . . . . . . . . 15
- 10 Analyzing Discussion During the Recovery Phase of Hurricane Sandy . . 16
- 1 A Comparison of the Tweet Collection Rate and the Tweet Processing Rate in the Random Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
- 1 User Visualization of Geo-Relevancy and Topic Affinity for a Topic in Egypt. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
- 2 Group Embedding of Influentials and Q1 Users . . . . . . . . . . . . . . . . . . . . . . . 50
- 1 Tumblr Profile of BBC News . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
- 2 Distribution of Likes in Tumblr Blogs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 5.3 Effect of the Number of Special Characters on Popularity . . . . . . . . . . . . . 65
- 4 Effect of Numeric Sequence Length on Popularity . . . . . . . . . . . . . . . . . . . . 65
- 5 Effect of the Length of the Username on Popularity . . . . . . . . . . . . . . . . . . 66
- 6 Effect of the Length of the Blogname on Popularity . . . . . . . . . . . . . . . . . . 67
- 7 Effect of the Number of Unique Characters on Popularity . . . . . . . . . . . . . 67
- 8 Effect of Similarity Between Blogname and Username on Popularity . . . 68 ix 5.
- 9 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
- 1 Distribution of Client Usage in Crisis Data . . . . . . . . . . . . . . . . . . . . . . . . . . 83
- 2 Crisis Data Generated from the Top 100 Clients . . . . . . . . . . . . . . . . . . . . . . 83
- 3 Average Probability of Tweets in Various Crises Data . . . . . . . . . . . . . . . . 92
- 4 A Comparison of the F 1 Score . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
- 5 A Comparison of the Weighted AUC Score . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 x Chapter 1 REFERENCES Aggarwal, C. C. and K. Subbian, "Event Detection in Social Streams", in "SDM", pp. 624-635 (2012). 3.5
- Agichtein, E., C. Castillo, D. Donato, A. Gionis and G. Mishne, "Finding High- Quality Content in Social Media", in "Proceedings of the 2008 International Con- ference on Web Search and Data Mining", pp. 183-194 (ACM, 2008). 5.6.3
- Allan, J., J. Carbonell, G. Doddington, J. Yamron and Y. Yang, "Topic Detection and Tracking Pilot Study Final Report", Tech. rep. (1998a). 3.1, 3.5
- Allan, J., R. Papka and V. Lavrenko, "On-Line New Event Detection and Track- ing", in "Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval", pp. 37-45 (ACM, 1998b).
- Aura, S. and G. D. Hess, "What's in a Name?", Economic Inquiry 48, 1, 214-227 (2010). 5.4
- Backstrom, L., E. Sun and C. Marlow, "Find Me If You Can: Improving Geograph- ical Prediction with Social and Spatial Proximity", in "Proceedings of the 19th International Conference on World Wide Web", pp. 61-70 (2010). 6.5
- Baddeley, A. D., N. Thomson and M. Buchanan, "Word Length and the Structure of Short-Term Memory", Journal of Verbal Learning and Verbal Behavior 14, 6, 575-589 (1975). 5.6.1
- Bandari, R., S. Asur and B. A. Huberman, "The Pulse of News in Social Media: Forecasting Popularity", in "Proceedings of 6th AAAI International Conference on Weblogs and Social Media", (The AAAI Press, 2012). 5.4
- Barbier, G., R. Zafarani, H. Gao, G. Fung and H. Liu, "Maximizing Benefits from Crowdsourced Data", Computational & Mathematical Organization Theory pp. 1-23 (2012). 4.6
- Bastian, M., S. Heymann and M. Jacomy, "Gephi: An Open Source Software for Exploring and Manipulating Networks", in "Proceedings of 3rd AAAI International Conference on Weblogs and Social Media", vol. 2 (The AAAI Press, 2009). 2.4
- Baur, M. and T. Schank, Dynamic Graph Drawing in Visone (Univ., Fak. für Infor- matik, 2008). 2.4
- Becker, H., M. Naaman and L. Gravano, "Learning Similarity Metrics for Event Identification in Social Media", in "Proceedings of the Third ACM International Conference on Web Search and Data Mining", pp. 291-300 (ACM, 2010). 3.5
- Blei, D., A. Ng and M. Jordan, "Latent Dirichlet Allocation", JMLR 3, 993-1022 (2003). 4.4.1
- Bowman, S. and C. Willis, "We Media: How Audiences Are Shaping the Future of News and Information", The Media Center at the American Press Institute (2003).
- Brandes, U., P. Kenis and J. Raab, "Explanation Through Network Visualization", Methodology: European Journal of Research Methods for the Behavioral and Social Sciences 2, 1, 16-23 (2006). 2.4
- Carley, K. M., J. Pfeffer, J. Reminga, J. Storrick and D. Columbus, "ORA User's Guide 2013", Tech. rep., DTIC Document (2013). 2.4
- Castillo, C., M. Mendoza and B. Poblete, "Information credibility on twitter", in "Proceedings of the 20th International Conference on World Wide Web", pp. 675- 684 (ACM, 2011). 6.2.3
- Cataldi, M., L. Di Caro and C. Schifanella, "Emerging Topic Detection on Twitter Based on Temporal and Social Terms Evaluation", in "Proceedings of the Tenth Int. Workshop on Multimedia Data Mining", p. 4 (2010). 4.6
- Chen, S. F. and J. Goodman, "An Empirical Study of Smoothing Techniques for Language Modeling", Computer Speech & Language 13, 4, 359-393 (1999). 5.6.3
- Cheng, Z., J. Caverlee and K. Lee, "You Are Where You Tweet: A Content-Based Approach to Geo-Locating Twitter Users", in "Proceedings of the 19th ACM In- ternational Conference on Information and Knowledge Management", pp. 759-768 (2010). 6.1, 6.3.1
- Cheng, Z., J. Caverlee and K. Lee, "A Content-driven Framework for Geolocating Microblog Users", ACM Trans. Intell. Syst. Technol. 4, 1, 2:1-2:27 (2013). 6.1, 6.5
- Cho, E., S. A. Myers and J. Leskovec, "Friendship and mobility: User Movement in Location-Based Social Networks", in "Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining", pp. 1082- 1090 (2011). 6.1
- Chua, F. C. T. and S. Asur, "Automatic Summarization of Events From Social Me- dia", Technical Report (2013). 6.4
- Cover, T., J. Thomas et al., Elements of Information Theory (Wiley Online Library, 1991). 4.5.3
- Dykes, J., J. Wood and A. Slingsby, "Rethinking Map Legends with Visualization", IEEE Transactions on Visualization and Computer Graphics 16, 6, 890-899 (2010).
- Endres, D. and J. Schindelin, "A New Metric for Probability Distributions", IEEE Transactions on Information Theory 49, 7, 1858-1860 (2003). 4.5.3
- Ferreira, N., L. Lins, D. Fink, S. Kelling, C. Wood, J. Freire and C. Silva, "BirdVis: Visualizing and Understanding Bird Populations", IEEE Transactions on Visual- ization and Computer Graphics 17, 12, 2374 -2383 (2011). 2.4
- Figlio, D. N., "Names, Expectations and the Black-White Test Score Gap", Tech. rep., National Bureau of Economic Research (2005). 5.4
- Fung, G. P. C., J. X. Yu, P. S. Yu and H. Lu, "Parameter Free Bursty Events Detection in Text Streams", in "Proceedings of the 31st International Conference on Very Large Data Bases", pp. 181-192 (VLDB Endowment, 2005). 3.4.1, 3.5
- Gao, H., G. Barbier and R. Goolsby, "Harnessing the Crowdsourcing Power of Social Media for Disaster Relief", Intelligent Systems, IEEE 26, 3, 10-14 (2011). 4.1, 4.6
- Guskin, Emily and Hitlin, Paul, "Hurricane Sandy and Twitter", http: //www.journalism.org/2012/11/06/hurricane-sandy-and-twitter/, [Online; accessed 4-December-2014] (2012). 1
- Hall, M., E. Frank, G. Holmes, B. Pfahringer, P. Reutemann and I. H. Witten, "The WEKA Data Mining Software: An Update", SIGKDD Explorations Newsletter pp. 10-18 (2009). 5.7.1, 6.3.1
- Hecht, B., L. Hong, B. Suh and E. H. Chi, "Tweets From Justin Bieber's Heart: The Dynamics of the Location Field in User Profiles", in "Proceedings of the SIGCHI Conference on Human Factors in Computing Systems", pp. 237-246 (2011). 6.5
- Heverin, T. and L. Zach, "Microblogging for Crisis Communication: Examination of Twitter Use in Response to a 2009 Violent Crisis in the Seattle-Tacoma, Washing- ton, Area", in "Proceedings of the 7th International ISCRAM Conference", vol. 1 (2010). 6.2.2
- Heylighen, F. and J.-M. Dewaele, "Variation in the Contextuality of Language: An Empirical Measure", Foundations of Science 7, 3, 293-340 (2002). 5.6.3
- Hong, L. and B. Davison, "Empirical study of topic modeling in twitter", in "Proceed- ings of the First Workshop on Social Media Analytics", pp. 80-88 (ACM, 2010).
- Hu, X., L. Tang, J. Tang and H. Liu, "Exploiting Social Relations for Sentiment Anal- ysis in Microblogging", in "Proceedings of the 6th ACM International Conference on Web Search and Data Mining", pp. 537-546 (2013). 6.2.4
- Hughes, A. L. and L. Palen, "Twitter Adoption and Use in Mass Convergence and Emergency Events", International Journal of Emergency Management 6, 3, 248- 260 (2009a). 5.1
- Hughes, A. L. and L. Palen, "Twitter Adoption and Use in Mass Convergence and Emergency Events", International Journal of Emergency Management 6, 3, 248- 260 (2009b). 6.2.2
- Kalist, D. E. and D. Y. Lee, "First Names and Crime: Does Unpopularity Spell Trouble?", Social Science Quarterly 90, 1, 39-49 (2009). 5.4
- Keogh, E., S. Lonardi and C. Ratanamahatana, "Towards Parameter-Free Data Min- ing", in "Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining", pp. 206-215 (ACM, 2004). 3.3.1
- Khamadi Were, D., "How Kenya turned to social media after mall attack", http://edition.cnn.com/2013/09/25/opinion/kenya-social-media-attack/ index.html?hpt=hp_c4, [Online; accessed 27-January-2014] (2013). 1, 3.1
- Kim, S., "Twitter's IPO Filing Shows 215 Million Monthly Active Users", http://abcnews.go.com/Business/ twitter-ipo-filing-reveals-500-million-tweets-day/story?id=20460493, [Online; accessed 26-February-2014] (2013). 1
- Kireyev, K., L. Palen and K. Anderson, "Applications of Topics Models to Analysis of Disaster-Related Twitter Data", in "NIPS Workshop on Applications for Topic Models: Text and Beyond", (2009). 4.6
- Krause, A. and D. Golovin, "Submodular Function Maximization", Tractability Prac- tical Approaches to Hard Problems 3 (2012). 3.2.1
- Kumar, S., G. Barbier, M. A. Abbasi and H. Liu, "TweetTracker: An Analysis Tool for Humanitarian and Disaster Relief.", in "Proeedings of 5th AAAI International Conference on Weblogs and Social Media", (The AAAI Press, 2011a). 2.1, 4.6
- Kumar, S., F. Morstatter, R. Zafarani and H. Liu, "Whom Should I Follow? Identi- fying Relevant Users During Crises", in "Proceedings of the 24th ACM Conference on Hypertext and Social Media", pp. 139-147 (ACM, 2013).
- Kumar, S., R. Zafarani and H. Liu, "Understanding User Migration Patterns in Social Media", in "AAAI", (2011b).
- Kunegis, J., A. Lommatzsch and C. Bauckhage, "The Slashdot Zoo: Mining a Social Network with Negative Edges", in "Proceedings of the 18th International Confer- ence on World Wide Web", pp. 741-750 (ACM, 2009). 5.4
- Kwak, H., C. Lee, H. Park and S. Moon, "What is Twitter, a Social Network or a News Media?", in "Proceedings of the 19th International Conference on World Wide Web", pp. 591-600 (ACM, 2010). 6.2.5
- Larkey, L. S. and M. E. Connell, "Arabic Information Retrieval at UMass in TREC- 10", Tech. Rep. ADA456273, University of Massachussetts (2006). 4.3
- Lerman, K. and T. Hogg, "Using a Model of Social Dynamics to Predict Popularity of News", in "Proceedings of the 19th International Conference on World Wide Web", pp. 621-630 (ACM, 2010). 5.4
- Li, C., A. Sun and A. Datta, "Twevent: Segment-Based Event Detection from Tweets", in "Proceedings of the 21st ACM International Conference on Information and Knowledge Management", pp. 155-164 (ACM, 2012). 2.4, 3.5
- Lin, J., "Divergence Measures Based on the Shannon Entropy", IEEE Transactions on Information Theory 37, 1, 145-151 (1991). 4.5.3
- MacEachren, A., A. Jaiswal, A. Robinson, S. Pezanowski, A. Savelyev, P. Mitra, X. Zhang and J. Blanford, "SensePlace2: GeoTwitter Analytics Support for Sit- uational Awareness", in "2011 IEEE Conference on Visual Analytics Science and Technology (VAST)", pp. 181 -190 (2011). 2.4
- Mahmud, J., J. Nichols and C. Drews, "Where Is This Tweet From? Inferring Home Locations of Twitter Users", in "Proceedings of 6th AAAI International Conference on Weblogs and Social Media", (The AAAI Press, 2012). 6.1, 6.3.1, 6.5
- Manning, C. D., P. Raghavan and H. Schütze, Introduction to Information Retrieval, vol. 1 (Cambridge university press Cambridge, 2008). 5.7.1
- Mathioudakis, M. and N. Koudas, "TwitterMonitor: Trend Detection over the Twit- ter Stream", in "SIGMOD", pp. 1155-1158 (ACM, 2010). 2.4, 4.6
- Mendoza, M., B. Poblete and C. Castillo, "Twitter Under Crisis: Can We Trust What We RT?", in "Proceedings of the First Workshop on Social Media Analytics", pp. 71-79 (2010). 3.1, 3.4.1, 4.1, 4.6, 6.5
- Miller, R. and N. Lammas, "Social Media and its Implications for Viral Marketing", Asia Pacific Public Relations Journal 11, 1, 1-9 (2010). 5.1
- Morstatter, F., N. Lubold, H. Pon-Barry, J. Pfeffer and H. Liu, "Finding Eyewitness Tweets During Crises", in "Workshop on Language Technology and Computational Social Science", (2014). 6.5
- Morstatter, F., J. Pfeffer, H. Liu and K. M. Carley, "Is the Sample Good Enough? Comparing Data from Twitter's Streaming API with Twitter's Firehose", Proceed- ings of 7th AAAI International Conference on Weblogs and Social Media (2013).
- NHC, "Post-Tropical Cyclone SANDY", http://www.nhc.noaa.gov/archive/ 2012/al18/al182012.update.10300002.shtml (2012). 2.5.1
- Nocke, T., M. Flechsig and U. Bohm, "Visual Exploration and Evaluation of Climate- Related Simulation Data", in "Simulation Conference, 2007 Winter", pp. 703-711 (IEEE, 2007). 2.4
- Owoputi, O., B. O'Connor, C. Dyer, K. Gimpel, N. Schneider and N. A. Smith, "Im- proved Part-Of-Speech Tagging for Online Conversational Text with Word Clus- ters", in "Proceedings of NAACL-HLT", pp. 380-390 (2013). 5.6.3, 6.2.4
- Page, R., "The Linguistics of Self-Branding and Micro-Celebrity in Twitter: The Role of Hashtags", Discourse & Communication 6, 2, 181-201 (2012). 6.2.4
- Pal, A. and S. Counts, "Identifying Topical Authorities in Microblogs", in "Proceed- ings of the fourth ACM International Conference on Web Search and Data Mining", pp. 45-54 (ACM, 2011). 5.4
- Panagiotopoulos, P., A. Z. Bigdeli and S. Sams, ""5 Days in August"-How London Local Authorities Used Twitter during the 2011 Riots", in "Electronic Govern- ment", pp. 102-113 (Springer, 2012). 6.5
- Paris, C., P. Thomas and S. Wan, "Differences in language and style between two social media communities.", in "Proceedings of 6th AAAI International Conference on Weblogs and Social Media", (The AAAI Press, 2012). 1, 3.1
- Pear Analytics, "Twitter Study", http://www.pearanalytics.com/wp-content/ uploads/2012/12/Twitter-Study-August-2009.pdf, [Online; accessed 27- January-2014] (2009). 3.1
- Perreault, M. and D. Ruths, "The Effect of Mobile Platforms on Twitter Content Generation", in "Proceedings of 5th AAAI International Conference on Weblogs and Social Media", (The AAAI Press, 2011). 6.2.2
- Petrovic, S., M. Osborne and V. Lavrenko, "Streaming First Story Detection with Application to Twitter", in "Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics", vol. 10 (Citeseer, 2010). 3.4.1, 3.5, 3.6
- Poell, T. and E. Borra, "Twitter, YouTube, and Flickr as Platforms of Alternative Journalism: The Social Media Account of the 2010 Toronto G20 Protests", Jour- nalism 13, 6, 695-713 (2012). 5.5.1
- Popescu, A. and M. Pennacchiotti, "Detecting Controversial Events from Twitter", in "CIKM", pp. 1873-1876 (2010). 4.6
- Purohit, H. and A. P. Sheth, "Twitris v3: From Citizen Sensing to Analysis, Coor- dination and Action", in "Proceedings of 7th AAAI International Conference on Weblogs and Social Media", (The AAAI Press, 2013). 2.4
- Qu, Y., C. , P. Zhang and J. Zhang, "Microblogging After a Major Disaster in China: A Case Study of the 2010 Yushu Earthquake", in "Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work", pp. 25-34 (2011). 3.1, 4.6
- Ramage, D., S. Dumais and D. Liebling, "Characterizing Microblogs with Topic Mod- els", in "Proceedings of 4th AAAI International Conference on Weblogs and Social Media", (The AAAI Press, 2010). 4.6
- Rijsbergen, C. J. V., Information Retrieval (Butterworth-Heinemann, Newton, MA, USA, 1979), 2nd edn. 3.3
- Rout, D., K. Bontcheva, D. Preotiuc-Pietro and T. Cohn, "Where's @Wally?: a Classification Approach to Geolocating Users Based on their Social Ties.", in "Pro- ceedings of the 24th ACM Conference on Hypertext and Social Media", pp. 11-20
- Russ, H., "New York, New Jersey Put $71B Price Tag on Sandy", http://news. msn.com/us/new-york-new-jersey-put-dollar71b-price-tag-o n-sandy (2012). 2.5.1
- Sakaki, T., M. Okazaki and Y. Matsuo, "Earthquake Shakes Twitter Users: Real- Time Event Detection by Social Sensors", in "Proceedings of the 19th International Conference on World Wide Web", pp. 851-860 (2010). 3.1, 3.4.1, 3.5, 4.1, 4.6, 6.5
- Sayyadi, H., M. Hurst and A. Maykov, "Event Detection and Tracking in Social Streams", in "Proceedings of 3rd AAAI International Conference on Weblogs and Social Media", (The AAAI Press, 2009). 3.5
- Shannon, P., A. Markiel, O. Ozier, N. S. Baliga, J. T. Wang, D. Ramage, N. Amin, B. Schwikowski and T. Ideker, "Cytoscape: A Software Environment for Integrated Models of Biomolecular Interaction Networks", Genome research 13, 11, 2498-2504 (2003). 2.4
- Shirky, C., "Power Laws, Weblogs, and Inequality", Clay Shirky's writings about the Internet 8 (2003). 5.4, 5.5.1
- Sinnappan, S., C. Farrell and E. Stewart, "Priceless Tweets! A Study on Twitter Messages Posted During Crisis: Black Saturday", in "ACIS 2010 Proceedings", (AIS Electronic Library, 2010). 6.5
- Tenenbaum, J., V. De Silva and J. Langford, "A Global Geometric Framework for Nonlinear Dimensionality Reduction", Science 290, 5500, 2319-2323 (2000). 4.5.3
- Verma, S., S. Vieweg, W. J. Corvey, L. Palen, J. H. Martin, M. Palmer, A. Schram and K. M. Anderson, "Natural Language Processing to the Rescue? Extracting "Situational Awareness" Tweets During Mass Emergency", in "Proceedings of 5th AAAI International Conference on Weblogs and Social Media", (The AAAI Press, 2011). 6.3.1
- Weng, J. and B. S. Lee, "Event detection in twitter", in "Proceedings of 5th AAAI International Conference on Weblogs and Social Media", (The AAAI Press, 2011).
- Wikipedia, "Earthquakes in 2011", http://en.wikipedia.org/wiki/Earthquakes_ in_2011, [Online; accessed 27-January-2014] (2011). 3.4.1
- Wikipedia, "Earthquakes in 2012", http://en.wikipedia.org/wiki/Earthquakes_ in_2012, [Online; accessed 27-January-2014] (2012). 3.4.1
- Wikipedia, "Venezuelan Presidential Election, 2013", http://en.wikipedia.org/ wiki/Venezuelan_presidential_election,_2013, [Online; accessed 27-January- 2014] (2013). 3.4.2
- Yang, Y., T. Pierce and J. Carbonell, "A Study of Retrospective and On-Line Event Detection", in "Proceedings of the 21st Annual International ACM SIGIR Confer- ence on Research and Development in Information Retrieval", pp. 28-36 (ACM, 1998). 3.1, 3.4.1, 3.5
- Ye, J., J.-H. Chow, J. Chen and Z. Zheng, "Stochastic Gradient Boosted Distributed Decision Trees", in "Proceedings of the 18th ACM Conference on Information and Knowledge Management", CIKM '09, pp. 2061-2064 (ACM, New York, NY, USA, 2009). 5.7.1
- Zhao, Q., P. Mitra and B. Chen, "Temporal and Information Flow Based Event Detection From Social Text Streams", in "AAAI", vol. 7, pp. 1501-1506 (2007).
- Zhao, W., J. Jiang, J. Weng, J. He, E. Lim, H. Yan and X. Li, "Comparing Twitter and Traditional Media Using Topic Models", Advances in Information Retrieval pp. 338-349 (2011). 4.6