Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Background
Related Work
System Overview
Analysis
Representing Summary Content
Aggregating Summary Content
Corpus Analysis
Conclusion and Future Work
References
All Topics
Psychology
Cognitive Science

Summarizing Information Graphics Textually

Profile image of Kathleen McCoyKathleen McCoy

2012, Computational Linguistics

https://doi.org/10.1162/COLI_A_00091
visibility

…

description

48 pages

link

1 file

Abstract

Information graphics (such as bar charts and line graphs) play a vital role in many multimodal documents. The majority of information graphics that appear in popular media are intended to convey a message and the graphic designer uses deliberate communicative signals, such as highlighting certain aspects of the graphic, in order to bring that message out. The graphic, whose communicative goal (intended message) is often not captured by the document's accompanying text, contributes to the overall purpose of the document and cannot be ignored. This article presents our approach to providing the high-level content of a non-scientific information graphic via a brief textual summary which includes the intended message and the salient features of the graphic. This work brings together insights obtained from empirical studies in order to determine what should be contained in the summaries of this form of non-linguistic input data, and how the information required for realizing the sele...

Related papers

Adapting Graph Summaries to the Users’ Reading Levels
Priscilla Moraes

Proceedings of the 8th International Natural Language Generation Conference (INLG), 2014

Deciding on the complexity of a generated text in NLG systems is a contentious task. Some systems propose the generation of simple text for low-skilled readers; some choose what they anticipate to be a "good measure" of complexity by balancing sentence length and number of sentences (using scales such as the D-level sentence complexity) for the text; while others target high-skilled readers. In this work, we discuss an approach that aims to leverage the experience of the reader when reading generated text by matching the syntactic complexity of the generated text to the reading level of the surrounding text. We propose an approach for sentence aggregation and lexical choice that allows generated summaries of line graphs in multimodal articles available online to match the reading level of the text of the article in which the graphs appear. The technique is developed in the context of the SIGHT (Summarizing Information Graphics Textually) system. This paper tackles the micro planning phase of sentence generation discussing additionally the steps of lexical choice, and pronominalization.

View PDFchevron_right
Towards retrieving relevant information graphics
Kathleen McCoy

Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval, 2013

Information retrieval research has made significant progress in the retrieval of text documents and images. However, relatively little attention has been given to the retrieval of information graphics (non-pictorial images such as bar charts and line graphs) despite their proliferation in popular media such as newspapers and magazines. Our goal is to build a system for retrieving bar charts and line graphs that reasons about the content of the graphic itself in deciding its relevance to the user query. This paper presents the first steps toward such a system, with a focus on identifying the category of intended message of potentially relevant bar charts and line graphs. Our learned model achieves accuracy higher than 80% on a corpus of collected user queries.

View PDFchevron_right
Towards generating textual summaries of graphs
Kathleen McCoy

Hci, 2001

For people who use text-based web browsers, graphs, diagrams, and pictures are inaccessible. Yet, such diagrams are quite prominent in documents commonly found on the web. In this work we describe a method for rendering important aspects of a line graph as plain text. Our method relies heavily on research in natural language generation and is motivated by an analysis of human-written descriptions of line graphs. In this paper we concentrate on determining which aspects of the line graph to include in a summary.

View PDFchevron_right
Understanding information graphics: A discourse-level problem
Kathleen McCoy

Proceedings of SIGDial, 2003

Information graphics that appear in newspapers and magazines generally have a message that the viewer is intended to recognize. This paper argues that understanding such information graphics is a discourse-level problem. In particular, it requires assimilating information from multiple knowledge sources to recognize the intended message of the graphic, just as recognizing intention in text does. Moreover, when an article is composed of text and graphics, the intended message of the information graphic (its discourse intention) must be integrated into the discourse structure of the surrounding text and contributes to the overall discourse intention of the article. This paper describes how we extend plan-based techniques that have been used for understanding traditional discourse to the understanding of information graphics. This work is part of a project to develop an interactive natural language system that provides sight-impaired users with access to information graphics.

View PDFchevron_right
Towards Understanding How Readers Integrate Charts and Captions: A Case Study with Line Charts
Vidya Setlur

Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021

Charts often contain visually prominent features that draw attention to aspects of the data and include text captions that emphasize aspects of the data. Through a crowdsourced study, we explore how readers gather takeaways when considering charts and captions together. We frst ask participants to mark visually prominent regions in a set of line charts. We then generate text captions based on the prominent features and ask participants to report their takeaways after observing chart-caption pairs. We fnd that when both the chart and caption describe a high-prominence feature, readers treat the doubly emphasized high-prominence feature as the takeaway; when the caption describes a low-prominence chart feature, readers rely on the chart and report a higher-prominence feature as the takeaway. We also fnd that external information that provides context, helps further convey the caption's message to the reader. We use these fndings to provide guidelines for authoring efective chart-caption pairs. CCS CONCEPTS • Human-centered computing → Empirical studies in visualization.

View PDFchevron_right
When Does an Infographic Say More Than a Thousand Words?
Yael de Haan

Journalism Studies, 2017

In recent years, the use of visualizations or infographics in the news has become increasingly popular. We know, however, surprisingly little about how news consumers use and appreciate news visualizations. We apply a mixed-method approach to answer these two questions. First, we conduct an eye-tracking study that measures use, by means of direct attention to visualizations on three different news platforms (print newspaper, e-newspaper on tablet, and news website). Second, we conduct focus groups and a survey among readers of three news media to study the extent to which news consumers actually value the inclusion of visualizations in the news. Our results show that news consumers do indeed read news visualizations, regardless of the platform on which the visual is published. We also find that visualizations are appreciated, but only if they are coherently integrated into a news story and thus fulfill a function that can be easily understood. With this study, we provide the first comprehensive picture of the usefulness of information visualizations in the news, and contribute to a growing literature on alternative ways of storytelling in journalism today.

View PDFchevron_right
Improving the Accessibility of Line Graphs in Multimodal Documents
David McDonald

2011

This paper describes our work on improving access to the content of multimodal documents containing line graphs in popular media for people with visual impairments. We provide an overview of our implemented system, including our method for recognizing and conveying the intended message of a line graph. The textual description of the graphic generated by our system is presented at the most relevant point in the document. We also describe ongoing work into obtaining additional propositions that elaborate on the intended message, and examine the potential benefits of analyzing the text and graphical content together in order to extend our system to produce summaries of entire multimodal documents.

View PDFchevron_right
Fundamentals of the visual language in the composition of infographics
Luciano Adorno

2010

The objective of this paper is to defend and to justify the relevance of the fundamentals of the visual language to composition of infographics. It is considered as basic elements of infographic: points, lines, planes and spots, and they are the same ones that characterize the composition and visual communication from prehistory. These same elements make up the expressiveness of designs and Design products. In the text, reflections are presented about Form’s Theory and Visual Language, based on Lupton and Phillips, Dondis and Perassi. Besides the presentation of concepts, historical datas and discussions on infographics, based on Ribeiro, Rossi and others authors.

View PDFchevron_right
Towards constructive text, diagram, and layout generation for information presentation
Klaus Reichenberger, John Bateman

Computational …, 2001

View PDFchevron_right
Pay "Attention" to Chart Images for What You Read on Text
Xiaoman Zhao

Companion of the 2023 International Conference on Management of Data

Data visualization is changing how we understand data, by showing why's, how's, and what's behind important patterns/trends in almost every corner of the world, such as in academic papers, news articles, financial reports, etc. However, along with the increasing complexity and richness of data visualization, given a text description (e.g., "fewer teens say they attended school completely online (8%)"), it becomes harder for users to pinpoint where to pay attention to on a chart (e.g., a grouped bar chart). In this demonstration paper, we present a system HiChart for text-chart image highlighting: when a user selects a span of text, HiChart automatically analyzes the chart image (e.g., a jpeg or a png file) and highlights the parts that are relevant to the span. From a technical perspective, HiChart devises the following techniques. Reverse-engineering visualizations: given a chart image, HiChart uses computer vision techniques to generate a visualization specification using Vega-Lite language, as well as the underlying dataset; Visualization calibration by data tuning: HiChart calibrates the regenerated chart by tuning the recovered dataset through value perturbation; and Chart highlighting for a span: HiChart maps the span to corresponding data cells and uses the built-in highlighting functions of Vega-Lite to highlight the chart. CCS CONCEPTS • Human-centered computing → Visualization.

View PDFchevron_right

References (82)

  1. Alty, James L. and Dimitrios I. Rigas. 1998. Communicating graphical information to blind users using music: the role of context. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 574-581, Los Angeles, CA.
  2. Baldwin, Breck and Thomas Morton. 1998. Dynamic coreference-based summarization. In Proceedings of the 3rd Conference on Empirical Methods in Natural Language Processing, pages 1-6, Granada.
  3. Barzilay, Regina, Noemie Elhadad, and Kathleen McKeown. 2002. Inferring strategies for sentence ordering in multidocument news summarization. Journal of Artificial Intelligence Research, 17:35-55.
  4. Barzilay, Regina and Mirella Lapata. 2006. Aggregation via set partitioning for natural language generation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, pages 359-366, New York, NY.
  5. Belz, Anja, Eric Kow, Jette Viethen, and Albert Gatt. 2008. The Grec challenge 2008: Overview and evaluation results. In Proceedings of the 5th International Natural Language Generation Conference, pages 183-191, Salt Fork, OH.
  6. Brennan, Susan E., Marilyn W. Friedman, and Carl J. Pollard. 1987. A centering approach to pronouns. In Proceedings of the Annual Meeting of the Association of Computational Linguistics, pages 155-162, Stanford, CA.
  7. Carberry, Sandra, Stephanie Elzer, and Seniz Demir. 2006. Information graphics: An untapped resource for digital libraries. In Proceedings of the ACM Special Interest Group on Information Retrieval Conference, pages 581-588, Seattle, WA.
  8. Chester, Daniel and Stephanie Elzer. 2005. Getting computers to see information graphics so users do not have to. In Proceedings of the 15th International Symposium on Methodologies for Intelligent Systems, pages 660-668, Saratoga Springs, NY.
  9. Clark, Herbert. 1996. Using Language. Cambridge University Press, Cambridge.
  10. Coch, Jose. 1998. Interactive generation and knowledge administration in multimeteo. In Proceedings of 9th International Workshop on Natural Language Generation, pages 300-303, Niagara-on-the-Lake.
  11. Corio, Marc and Guy Lapalme. 1999. Generation of texts for information graphics. In Proceedings of the 7th European Workshop on Natural Language Generation, pages 49-58, Toulouse.
  12. Covington, M., C. He, C. Brown, L. Naci, and J. Brown. 2006. How complex is that sentence? A proposed revision of the rosenberg and abbeduto D-level scale. Research Report, Artificial Intelligence Center, University of Georgia. Cycorp. Open Cyc. 2011. http://www.cyc.com.
  13. Dale, Robert and Ehud Reiter. 1995. Computational interpretations of the gricean maxims in the generation of referring expressions. Cognitive Science, 19(2):233-263.
  14. Dalianis, Hercules. 1999. Aggregation in natural language generation. Computational Intelligence, 15(4):384-414.
  15. Demir, Seniz, Sandra Carberry, and Stephanie Elzer. 2009. Issues in Realizing the Overall Message of a Bar Chart, John Benjamins, 5th edition. Amsterdam, pages 311-320.
  16. Demir, Seniz, David Oliver, Edward Schwartz, Stephanie Elzer, Sandra Carberry, Kathleen F. McCoy, and Daniel Chester. 2010. Interactive sight: Textual access to simple bar charts. The New Review of Hypermedia and Multimedia, 16(3):245-279.
  17. Di Eugenio, Barbara, Davide Fossati, Dan Yu, Susan Haller, and Michael Glass. 2005. Aggregation improves learning: Experiments in natural language generation for intelligent tutoring systems. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 50-57, Ann Arbor, MI.
  18. Elhadad, M. and J. Robin. 1999. SURGE: A comprehensive plug-in syntactic realization component for text generation. Technical Report, Department of Computer Science, Ben Gurion University. Beersheba, Israel.
  19. Elzer, Stephanie, Sandra Carberry, and Ingrid Zukerman. 2011. The automated understanding of simple bar charts. Artificial Intelligence, 175(2):526-555.
  20. Elzer, Stephanie, Nancy Green, Sandra Carberry, and James Hoffman. 2006. A model of perceptual task effort for bar charts and its role in recognizing intention. International Journal on User Modeling and User-Adapted Interaction, 16(1):1-30.
  21. Fasciano, Massimo and Guy Lapalme. 2000. Intentions in the coordinated generation of graphics and text from tabular data. Knowledge and Information Systems, 2(3):310-339.
  22. Fellbaum, Christiane. 1998. WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA.
  23. Ferres, Leo, Petro Verkhogliad, Gitte Lindgaard, Louis Boucher, Antoine Chretien, and Martin Lachance. 2007. Improving accessibility to statistical graphs: the igraph-lite system. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility, pages 67-74, Tempe, AZ.
  24. Foster, Mary Ellen. 1999. Automatically generating text to accompany information graphics. Master's Thesis, University of Toronto.
  25. Friendly, Michael. 2008. A brief history of data visualization. In C. Chen, W. Härdle, and A. Unwin, editors, Handbook of Computational Statistics: Data Visualization, volume III. Springer-Verlag, Heidelberg, pages 1-34.
  26. Gatt, Albert, Francois Portet, Ehud Reiter, Jim Hunter, Saad Mahamood, Wendy Moncur, and Somayajulu Sripada. 2009. From data to text in the neonatal intensive care unit: Using NLG technology for decision support and information management. AI Communications, 22(3):153-186.
  27. Goldberg, Eli, Norbert Driedger, and Richard I. Kittredge. 1994. Using natural-language processing to produce weather forecasts. IEEE Expert: Intelligent Systems and Their Applications, 9(2):45-53.
  28. Goldstein, Jade, Vibhu Mittal, Jaime Carbonell, and Mark Kantrowitz. 2000. Multi-document summarization by sentence extraction. In Proceedings of the NAACL-ANLP Workshop on Automatic Summarization, pages 40-48, Seattle, WA.
  29. Greenbacker, Charlie, Sandra Carberry, and Kathleen F. McCoy. 2011. A corpus of human-written summaries of line graphs. In Proceedings of the EMNLP 2011 Workshop on Language Generation and Evaluation (UCNLG+Eval), pages 23-27, Edinburgh.
  30. Grice, H. Paul. 1975. Logic and conversation. Speech Acts, 3:41-58.
  31. Grosz, Barbara and Candace Sidner. 1986. Attention, intentions, and the structure of discourse. Computational Linguistics, 12(3):175-204.
  32. Grosz, Barbara J., Scott Weinstein, and Aravind K. Joshi. 1995. Centering: A framework for modeling the local coherence of discourse. Computational Linguistics, 21(2):203-225.
  33. Hovy, Eduard H. 1988. Planning coherent multisentential text. In Proceedings of the 26th Annual Meeting of the Association for Computational Linguistics, pages 163-169, Buffalo, NY.
  34. Hovy, Eduard H. 1993. Automated discourse generation using discourse structure relations. Artificial Intelligence, 63(1-2):341-385.
  35. Hovy, Eduard and Chin-Yew Lin. 1996. Automated text summarization and the summarist system. In Proceedings of the Workshop on TIPSTER Text Program, pages 197-214, Vienna, VA.
  36. Ina, Satoshi. 1996. Computer graphics for the blind. SIGCAPH Computers and the Physically Handicapped, 55:16-23.
  37. Jayant, Chandrika, Matt Renzelmann, Dana Wen, Satria Krisnandi, Richard Ladner, and Dan Comden. 2007. Automated tactile graphics translation: in the field. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility, pages 75-82, Tempe, AZ.
  38. Johnson, Mark. 1998. Proof nets and the complexity of processing center embedded constructions. Journal of Logic, Language and Information, 7(4):433-447.
  39. Joshi, Aravind, Bonnie Webber, and Ralph Weischedel. 1984. Living up to expectations: Computing expert responses. In Proceedings of the National Conference on Artificial Intelligence, pages 169-175, Austin, TX.
  40. Karamanis, Nikiforos, Chris Mellish, Massimo Poesio, and Jon Oberlander. 2009. Evaluating centering for information ordering using corpora. Computational Linguistics, 35(1):29-46.
  41. Kennel, A. 1996. Audiograf: A diagram-reader for the blind. In Proceedings of the 2nd Annual ACM Conference on Assistive Technologies, pages 51-56, Vancover, BC, Canada.
  42. Kerpedjiev, Stephan and Steven Roth. 2000. Mapping communicative goals into conceptual tasks to generate graphics in discourse. In Proceedings of the International Conference on Intelligent User Interfaces, pages 60-67, New Orleans, LA.
  43. Kibble, Rodger and Richard Power. 2004. Optimizing referential coherence in text generation. Computational Linguistics, 30(4):401-416.
  44. Kidd, Evan and Edith Bavin. 2002. English-speaking children's comprehension of relative clauses: Evidence for general-cognitive and language-specific constraints on development. Journal of Psycholinguistic Research, 31(6):599-617.
  45. Krahmer, E. and M. Theune. 2002. Efficient context-sensitive generation of referring expressions. In K. van Deemter and R. Kibble, editors, Information Sharing: Reference and Presupposition in Language Generation and Interpretation, Center for the Study of Language and Information-Lecture Notes, volume 143 of CSLI Lecture Notes. CSLI Publications, Stanford, CA, pages 233-264.
  46. Krahmer, Emiel, Sebastiaan Van Erk, and André Verleg. 2003. Graph-based generation of referring expressions. Computational Linguistics, 29(1):53-72.
  47. Kukich, Karen. 1983. Design of a knowledge-based report generator. In Proceedings of the 21st Annual Meeting of the Association for Computational Linguistics, pages 145-150, Cambridge, MA.
  48. Kurze, Martin. 1995. Giving blind people access to graphics (example: business graphics). In Proceedings of the Software-Ergonomie Workshop, Dannstadt, Bremen.
  49. Lavoie, Benoit and Owen Rambow. 1997. A fast and portable realizer for text generation systems. In Proceedings of the 5th Conference on Applied Natural Language Processing, pages 265-268, Washington, DC.
  50. Lazar, J., A. Allen, J. Kleinman, and C. Malarkey. 2007. What frustrates screen reader users on the web: A study of 100 blind users. International Journal of Human-Computer Interaction, 22(3):247-269.
  51. Lester, James C. and Bruce W. Porter. 1997. Developing and empirically evaluating robust explanation generators: The KNIGHT experiments. Computational Linguistics, 23(1):65-101.
  52. Lin, Dekang. 1996. On the structural complexity of natural language sentences. In Proceedings of the International Conference on Computational Linguistics, pages 729-733, Copenhagen, Denmark.
  53. Mann, William C. and Sandra A. Thompson. 1987. Rhetorical structure theory: A theory of text organization. In Livia Polanyi, editor, The Structure of Discourse. Ablex Publishing Corporation, Norwood, NJ.
  54. Marcu, Daniel. 1998. The Rhetorical Parsing, Summarization, and Generation of Natural Language Texts. Ph.D. thesis, Department of Computer Science, University of Toronto.
  55. McCoy, Kathleen F., Sandra Carberry, Tom Roper, and Nancy Green. 2001. Towards generating textual summaries of graphs. In Proceedings of the 1st International Conference on Universal Access in Human- Computer Interaction, pages 695-699, New Orleans, LA.
  56. McCoy, Kathleen F. and Jeannette Cheng. 1991. Focus of attention: Constraining what can be said next. In Cecile Paris, William Swartout, and William Mann, editors, Natural Language Generation in Artificial Intelligence and Computational Linguistics. Kluwer Academic Publishers, Berlin, pages 103-124.
  57. McKeown, Kathleen R. 1985. Discourse strategies for generating natural-language text. Artificial Intelligence, 27(1):1-41.
  58. McKeown, Kathleen R., Shimei Pan, James Shaw, Desmond A. Jordan, and Barry A. Allen. 1997. Language generation for multimedia healthcare briefings. In Proceedings of the 5th Conference on Applied Natural Language Processing, pages 277-282, Washington, DC.
  59. Meijer, Peter B. 1992. An experimental system for auditory image representations. IEEE Transactions on Biomedical Engineering, 39(2):112-121.
  60. Mellish, Chris, Alisdair Knott, Jon Oberlander, and Mick O'Donnell. 1998. Experiments using stochastic search for text planning. In Proceedings of the 9th International Workshop on Natural Language Generation, pages 98-107, Niagara-on-the-Lake.
  61. Moore, Johanna D. and Cecile Paris. 1993. Planning text for advisory dialogues: Capturing intentional and rhetorical information. Computational Linguistics, 19(4):651-694.
  62. Nenkova, Ani and Kathleen McKeown. 2003. References to named entities: a corpus study. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, pages 70-72, Edmonton.
  63. O'Donnell, M., C. Mellish, J. Oberlander, and A. Knott. 2001. Ilex: an architecture for a dynamic hypertext generation system. Natural Language Engineering, 7(3):225-250.
  64. Paris, Cecile. 1988. Tailoring object descriptions to a user's level of expertise. Computational Linguistics, 14(3):64-78.
  65. Pastra, Katerina, Horacio Saggion, and Yorick Wilks. 2003. Extracting relational facts for indexing and retrieval of crime-scene photographs. Knowledge-Based Systems, 16(5-6):313-320.
  66. Portet, Francois, Ehud Reiter, Albert Gatt, Jim Hunter, Somayajulu Sripada, Yvonne Freer, and Cindy Sykes. 2009. Automatic generation of textual summaries from neonatal intensive care data. Artificial Intelligence, 173(7-8):789-816.
  67. Radev, Dragomir R., Hongyan Jing, Malgorzata Stys, and Daniel Tam. 2004. Centroid-based summarization of multiple documents. Information Processing and Management: An International Journal, 40(6):919-938.
  68. Ramloll, Rameshsharma, Wai Yu, Stephen Brewster, Beate Riedel, Mike Burton, and Gisela Dimigen. 2000. Constructing sonified haptic line graphs for the blind student: First steps. In Proceedings of the 4th International ACM Conference on Assistive Technologies, pages 17-25, Arlington, VA.
  69. Reiter, Ehud. 2007. An architecture for data-to-text systems. In Proceedings of the 11th European Workshop on Natural Language Generation, pages 97-104, Schloss Dagstuhl.
  70. Reiter, Ehud and Robert Dale. 2000. Building Natural-language Generation Systems. Cambridge University Press, Cambridge.
  71. Schiffman, Barry, Ani Nenkova, and Kathleen McKeown. 2002. Experiments in multidocument summarization. In Proceedings of the 2nd International Conference on Human Language Technology Research, pages 52-58, San Diego, CA.
  72. Shaw, James. 1998. Clause aggregation using linguistics knowledge. In Proceedings of the 9th International Workshop on Natural Language Generation, pages 138-147, Niagara-on-the-Lake.
  73. Somayajulu, Sripada, Ehud Reiter, and Ian Davy. 2003. Sumtime-mousam: Configurable marine weather forecast generator. Expert Update, 6(3):4-10.
  74. Stent, A., Rashmi Prasad, and Marilyn Walker. 2004. Trainable sentence planning for complex information presentation in spoken dialog systems. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, pages 79-86, Barcelona.
  75. Stone, Matthew and Bonnie Webber. 1998. Textual economy through closely coupled syntax and semantics. In Proceedings of the International Natural Language Generation Conference, pages 178-187, Niagara-on-the-Lake.
  76. Suri, Linda Z. and Kathleen F. McCoy. 1994. Raft/rapr and centering: A comparison and discussion of problems related to processing complex sentences. Computational Linguistics, 20(2):301-317.
  77. Turner, Ross, Yaji Sripada, and Ehud Reiter. 2009. Generating approximate geographic descriptions. In Proceedings of the 12th European Workshop on Natural Language Generation, pages 42-49, Athens.
  78. Walker, M., O. Rambow, and M. Rogati. 2002. Training a sentence planner for spoken dialogue using boosting. Computer Speech and Language: Special Issue on Spoken Language Generation, 16(3):409-434.
  79. Walker, M., A. Stent, F. Mairesse, and R. Prasad. 2007. Individual and domain adaptation in sentence planning for dialogue. Journal of Artificial Intelligence Research, 30(1):413-456.
  80. Wu, Peng, Sandra Carberry, Stephanie Elzer, and Daniel Chester. 2010. Recognizing the intended message of line graphs. In Proceedings of the International Conference on the Theory and Application of Diagrams, pages 220-234, Portland, OR.
  81. Yngve, Victor H. 1960. A model and an hypothesis for language structure. American Philosophical Society, 104:444-466.
  82. Yu, Jin, Ehud Reiter, Jim Hunter, and Chris Mellish. 2007. Choosing the content of textual summaries of large time-series data sets. Natural Engineering, 13(1):25-49.

Related papers

Extending Document Summarization to Information Graphics
Kathleen McCoy

Information graphics (non-pictorial graphics such as bar charts or line graphs) are an important component of multimedia documents. Often such graphics convey information that is not contained elsewhere in the document. Thus document summa- rization must be extended to include summarization of information graphics. This paper addresses our work on graphic summarization. It argues that the message that the graphic designer intended to con- vey must play a major role in determining the con- tent of the summary, and it outlines our approach to identifying this intended message and using it to construct the summary.

View PDFchevron_right
Generating textual summaries of bar charts
Kathleen McCoy

Proceedings of the Fifth International Natural Language Generation Conference on - INLG '08, 2008

Information graphics, such as bar charts and line graphs, play an important role in multimodal documents. This paper presents a novel approach to producing a brief textual summary of a simple bar chart. It outlines our approach to augmenting the core message of the graphic to produce a brief summary. Our method simultaneously constructs both the discourse and sentence structures of the textual summary using a bottom-up approach. The result is then realized in natural language. An evaluation study validates our generation methodology.

View PDFchevron_right
Abstractive summarization of line graphs from popular media
Kathleen McCoy

ACL HLT …, 2011

Information graphics (bar charts, line graphs, etc.) in popular media generally have a discourse goal that contributes to achieving the communicative intent of a multimodal document. This paper presents our work on abstractive summarization of line graphs. Our methodology involves hypothesizing the intended message of a line graph and using it as the core of a summary of the graphic. This core is then augmented with salient propositions that elaborate on the intended message.

View PDFchevron_right
What is being Measured in an Information Graphic?
Stephanie Schwartz

Lecture Notes in Computer Science, 2013

Information graphics (such as bar charts and line graphs) are widely used in popular media. The majority of such non-pictorial graphics have the purpose of communicating a high-level message which is often not repeated in the text of the article. Thus, information graphics together with the textual segments contribute to the overall purpose of an article and cannot be ignored. Unfortunately, information graphics often do not label the dependent axis with a full descriptor of what is being measured. In order to realize the high-level message of an information graphic in natural language, a referring expression for the dependent axis must be generated. This task is complex in that the required referring expression often must be constructed by extracting and melding pieces of information from the textual content of the graphic. Our heuristic-based solution to this problem has been shown to produce reasonable text for simple bar charts. This paper presents the extensibility of that approach to other kinds of graphics, in particular to grouped bar charts and line graphs. We discuss the set of component texts contained in these two kinds of graphics, how the methodology for simple bar charts can be extended to these kinds, and the evaluation of the enhanced approach.

View PDFchevron_right
Generating explanatory captions for information graphics
Giuseppe Carenini

1995

Graphical presentations can be used to communicate information in relational data sets succinctly and effectively. However, novel graphical presentations about numerous attributes and their relationships are often difficult to understand completely until explained. Automatically generated graphical presentations must therefore either be limited to simple, conventional ones, or risk incomprehensibility. One way of alleviating this problem is to design graphical presentation systems that can work in conjunction with a natural language generator to produce "explanatory captions." This paper presents three strategies for generating explanatory captions to accompany information graphics based on: (1) a representation of the structure of the graphical presentation (2) a framework for identifying the perceptual complexity of graphical elements, and (3) the structure of the data expressed in the graphic. We describe an implemented system and illustrate how it is used to generate explanatory captions for a range of graphics from a data set about real estate transactions in Pittsburgh.

View PDFchevron_right

Related topics

  • Cognitive Science
  • Computer Science
  • Computational Linguistics
  • Graphic Design
  • Evaluation Studies
  • Super Line Graph

    • Cited by

    Chart Question Answering: State of the Art and Future Directions
    AHMED MASRY

    Computer Graphics Forum

    Information visualizations such as bar charts and line charts are very common for analyzing data and discovering critical insights. Often people analyze charts to answer questions that they have in mind. Answering such questions can be challenging as they often require a significant amount of perceptual and cognitive effort. Chart Question Answering (CQA) systems typically take a chart and a natural language question as input and automatically generate the answer to facilitate visual data analysis. Over the last few years, there has been a growing body of literature on the task of CQA. In this survey, we systematically review the current state-of-the-art research focusing on the problem of chart question answering. We provide a taxonomy by identifying several important dimensions of the problem domain including possible inputs and outputs of the task and discuss the advantages and limitations of proposed solutions. We then summarize various evaluation techniques used in the surveyed papers. Finally, we outline the open challenges and future research opportunities related to chart question answering. CCS Concepts • Human-centered computing → Visualization; • Computing methodologies → Natural language processing; Automatic chart question answering is a challenging task because of the richness and ambiguities of natural language and complex reasoning that may be required to predict the answer [SBT * 16]. The problem is also very inter-disciplinary in nature, falling in the intersection of information visualization, natural language processing, and human computer interactions. Some early approaches on NLIs and chart question answering applied natural language processing techniques by largely depending on heuristics or grammar-based parsing techniques [SBT * 16, SS18, HSTD17, GDA * 15]. Recently, there have been also some attempts for applying deep learning models for understanding natural language

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved