Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Work
Dataset
Analysis Method
Results
Results and Evaluation
User Study
Participants
Procedure
Results and Discussion
Limitations and Future Work
Conclusion
References
All Topics
Computer Science
Data Science

EmphasisChecker: A Tool for Guiding Chart and Caption Emphasis

Profile image of Vidya SetlurVidya Setlur

2023, arXiv (Cornell University)

https://doi.org/10.48550/ARXIV.2307.13858
visibility

…

description

12 pages

link

1 file

Abstract

Recent work has shown that when both the chart and caption emphasize the same aspects of the data, readers tend to remember the doubly-emphasized features as takeaways; when there is a mismatch, readers rely on the chart to form takeaways and can miss information in the caption text. Through a survey of 280 chart-caption pairs in real-world sources (e.g., news media, poll reports, government reports, academic articles, and Tableau Public), we find that captions often do not emphasize the same information in practice, which could limit how effectively readers take away the authors' intended messages. Motivated by the survey findings, we present EMPHASISCHECKER, an interactive tool that highlights visually prominent chart features as well as the features emphasized by the caption text along with any mismatches in the emphasis. The tool implements a time-series prominent feature detector based on the Ramer-Douglas-Peucker algorithm and a text reference extractor that identifies time references and data descriptions in the caption and matches them with chart data. This information enables authors to compare features emphasized by these two modalities, quickly see mismatches, and make necessary revisions. A user study confirms that our tool is both useful and easy to use when authoring charts and captions.

Related papers

ChartAccent: Annotation for data-driven storytelling
Eun Choe

2017 IEEE Pacific Visualization Symposium (PacificVis)

Figure 1: Examples of annotated charts created with ChartAccent: (left) emphasizing the months when Charlotte and Seattle's temperatures are higher than New York's average; (right) the relationship between fertility rate and life expectancy, with text and image annotations for the United States, China, and India; countries from North and South America are highlighted in blue.

View PDFchevron_right
Describing Complex Charts in Natural Language: A Caption Generation System
Giuseppe Carenini

Computational Linguistics, 1998

Graphical presentations can be used to communicate information in relational data sets succinctly and effectively. However, novel graphical presentations that represent many attributes and relationships are often difficult to understand completely until explained. Automatically generated graphical presentations must therefore either be limited to generating simple, conventionalized graphical presentations, or risk incomprehensibility. A possible solution to this problem would be to

View PDFchevron_right
Generating explanatory captions for information graphics
Giuseppe Carenini

1995

Graphical presentations can be used to communicate information in relational data sets succinctly and effectively. However, novel graphical presentations about numerous attributes and their relationships are often difficult to understand completely until explained. Automatically generated graphical presentations must therefore either be limited to simple, conventional ones, or risk incomprehensibility. One way of alleviating this problem is to design graphical presentation systems that can work in conjunction with a natural language generator to produce "explanatory captions." This paper presents three strategies for generating explanatory captions to accompany information graphics based on: (1) a representation of the structure of the graphical presentation (2) a framework for identifying the perceptual complexity of graphical elements, and (3) the structure of the data expressed in the graphic. We describe an implemented system and illustrate how it is used to generate explanatory captions for a range of graphics from a data set about real estate transactions in Pittsburgh.

View PDFchevron_right
AutoCaption: An Approach to Generate Natural Language Description from Visualization Automatically
Liwenhan Xie, Yun Han, Datong Wei

2020 IEEE Pacific Visualization Symposium (PacificVis), 2020

In this paper, we propose a novel approach to generate captions for visualization charts automatically. In the proposed method, visual marks and visual channels, together with the associated text information in the original charts, are first extracted and identified with a multilayer perceptron classifier. Meanwhile, data information can also be retrieved by parsing visual marks with extracted mapping relationships. Then a 1-D convolutional residual network is employed to analyze the relationship between visual elements, and recognize significant features of the visualization charts, with both data and visual information as input. In the final step, the full description of the visual charts can be generated through a template-based approach. The generated captions can effectively cover the main visual features of the visual charts and support major feature types in commons charts. We further demonstrate the effectiveness of our approach through several cases.

View PDFchevron_right
Exploring and Exploiting the Limited Utility of Captions in Recognizing Intention in Information Graphics
Keith Trnka

2005

This paper presents a corpus study that explores the extent to which captions contribute to recognizing the intended message of an information graphic. It then presents an implemented graphic interpretation system that takes into account a variety of communicative signals, and an evaluation study showing that evidence obtained from shallow processing of the graphic's caption has a significant impact on the system's success. This work is part of a larger project whose goal is to provide sight-impaired users with effective access to information graphics.

View PDFchevron_right
Chart-to-Text: A Large-Scale Benchmark for Chart Summarization
AHMED MASRY

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Charts are commonly used for exploring data and communicating insights. Generating natural language summaries from charts can be very helpful for people in inferring key insights that would otherwise require a lot of cognitive and perceptual efforts. We present Chart-to-text, a large-scale benchmark with two datasets and a total of 44,096 charts covering a wide range of topics and chart types. We explain the dataset construction process and analyze the datasets. We also introduce a number of state-of-the-art neural models as baselines that utilize image captioning and data-totext generation techniques to tackle two problem variations: one assumes the underlying data table of the chart is available while the other needs to extract data from chart images. Our analysis with automatic and human evaluation shows that while our best models usually generate fluent summaries and yield reasonable BLEU scores, they also suffer from hallucinations and factual errors as well as difficulties in correctly explaining complex patterns and trends in charts. * Equal contribution. Listing order is based on the alphabetical ordering of author surnames.

View PDFchevron_right
Summarizing Information Graphics Textually
Kathleen McCoy

Computational Linguistics, 2012

Information graphics (such as bar charts and line graphs) play a vital role in many multimodal documents. The majority of information graphics that appear in popular media are intended to convey a message and the graphic designer uses deliberate communicative signals, such as highlighting certain aspects of the graphic, in order to bring that message out. The graphic, whose communicative goal (intended message) is often not captured by the document's accompanying text, contributes to the overall purpose of the document and cannot be ignored. This article presents our approach to providing the high-level content of a non-scientific information graphic via a brief textual summary which includes the intended message and the salient features of the graphic. This work brings together insights obtained from empirical studies in order to determine what should be contained in the summaries of this form of non-linguistic input data, and how the information required for realizing the sele...

View PDFchevron_right
Formalizing Emphasis in Information Visualization
Sheelagh Carpendale

Computer Graphics Forum, 2016

We provide a fresh look at the use and prevalence of emphasis effects in Infovis. Through a survey of existing emphasis frameworks, we extract a set-based approach that uses visual prominence to link visually and algorithmically diverse emphasis effects. Visual prominence provides a basis for describing, comparing and generating emphasis effects when combined with a set of general features of emphasis effects. Therefore, we use visual prominence and these general features to construct a new mathematical Framework for Information Visualization Emphasis, FIVE. The concepts we introduce to describe FIVE unite the emphasis literature and point to several new research directions for emphasis in information visualization.

View PDFchevron_right
Describing complex charts in natural language
Vibhu Mittal

Computational Linguistics, 1998

Graphical presentations can be used to communicate information in relational data sets succinctly and effectively. However, novel graphical presentations that represent many attributes and relationships are often difficult to understand completely until explained. Automatically generated graphical presentations must therefore either be limited to generating simple, conventionalized graphical presentations, or risk incomprehensibility. A possible solution to this problem would be to extend automatic graphical presentation systems to generate explanatory captions in natural language, to enable users to understand the information expressed in the graphic. This paper presents a system to do so. It uses a text planner to determine the content and structure of the captions based on: (1) a representation of the structure of the graphical presentation and its mapping to the data it depicts, (2) a framework for identi~ing the perceptual complexity of graphical elements, and (3) the structure of the data expressed in the graphic. The output of the planner is further processed regarding issues such as ordering, aggregation, centering, generating referring expressions, and lexical choice. We discuss the architecture of our system and its strengths and limitations, Our implementation is currently limited to 2-D charts and maps, but, except for lexical information, it is completely domain independent. We illustrate our discussion with yqgures and generated captions about housing sales in Pittsburgh.

View PDFchevron_right
Interpretive Impacts of Text Visualization
Geri Gay

ACM Transactions on Computer-Human Interaction, 2018

Information visualizations are often evaluated as a tool in terms of their ability to support performance of a specific task. This article argues that value can be gained by instead evaluating visualizations from a communicative perspective. Specifically, it explores how text visualization can influence the impacts that framing has on the perception of political issues. Using data from a controlled laboratory study, the results presented here demonstrate that exposure to a text visualization can mitigate framing effects. Furthermore, it also shows a transfer effect, where participants who saw the visualization remained uninfluenced by framing in subsequent texts, even when the visualization was absent. These results carry implications for the methods used to evaluate information visualization systems, for understanding the cognitive and interpretive mechanisms by which framing effects occur, and for exploring the design space of interactive text visualization.

View PDFchevron_right

References (74)

  1. S. K. Badam, Z. Liu, and N. Elmqvist. Elastic Documents: Coupling Text and Tables through Contextual Visualizations for Enhanced Document Reading. IEEE Transactions on Visualization and Computer Graphics, 25(1):661-671, 2019. doi: 10.1109/TVCG.2018.2865119 2
  2. L. Battle, P. Duan, Z. Miranda, D. Mukusheva, R. Chang, and M. Stone- braker. Beagle: Automated Extraction and Interpretation of Visualiza- tions from the Web. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1-8. ACM, New York, USA, 2018. doi: 10.1145/3173574.3174168 2, 9
  3. F. Beck and D. Weiskopf. Word-Sized Graphics for Scientific Texts. IEEE Transactions on Visualization and Computer Graphics, 23(6):1576-1587, 2017. doi: 10.1109/TVCG.2017.2674958 2
  4. A. D. Biderman. The Graph as a Victim of Adverse Discrimination and Segregation. Information Design Journal, 1(4):232-241, 1979. doi: 10. 1075/idj.1.4.03bid 2
  5. M. A. Borkin, Z. Bylinskii, N. W. Kim, C. M. Bainbridge, C. S. Yeh, D. Borkin, H. Pfister, and A. Oliva. Beyond Memorability: Visualiza- tion Recognition and Recall. IEEE Transactions on Visualization and Computer Graphics, 22(1):519-528, 2016. doi: 10.1109/TVCG.2015. 2467732 2
  6. J. Brooke. SUS: A 'Quick and Dirty' Usability Scale. In Usability Evalua- tion In Industry, chap. 21, pp. 189-194. Taylor & Francis Group, London, UK, 1996. doi: 10.1201/9781498710411-35 7
  7. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert- Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei. Language Models Are Few-Shot Learners. In Proceed- ings of the 34th International Conference on Neural Information Process- ing Systems. 159. Curran Associates Inc., Red Hook, USA, 2020. doi: 10 .48550/arXiv.2005.14165 9
  8. S. Carberry, S. Elzer, and S. Demir. Information Graphics: An Untapped Resource for Digital Libraries. In Proceedings of the 29th Annual Inter- national ACM SIGIR Conference on Research and Development in Infor- mation Retrieval, pp. 581-588. ACM, New York, USA, 2006. doi: 10. 1145/1148170.1148270 2
  9. A. X. Chang and C. Manning. SUTime: A Library for Recognizing and Normalizing Time Expressions. In Proceedings of the Eighth Inter- national Conference on Language Resources and Evaluation, pp. 3735- 3740. ELRA, Paris, France, 2012. 2, 5, 9
  10. C. Chen, R. Zhang, S. Kim, S. Cohen, T. Yu, R. Rossi, and R. Bunescu. Neural Caption Generation over Figures. In Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, pp. 482-485. ACM, New York, USA, 2019. doi: 10.1145/3341162.3345601 2
  11. C. Chen, R. Zhang, E. Koh, S. Kim, S. Cohen, T. Yu, R. Rossi, and R. Bunescu. Figure Captioning with Reasoning and Sequence-Level Training. ArXiv, abs/1906.02850, 2019. doi: 10.48550/arXiv.1906. 02850 2
  12. D. Chen and C. Manning. A Fast and Accurate Dependency Parser Using Neural Networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 740-750. ACL, Strouds- burg, USA, 2014. doi: 10.3115/v1/D14-1082 2, 6
  13. Z. Chen and H. Xia. CrossData: Leveraging Text-Data Connections for Authoring Data Documents. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 95. ACM, New York, USA, 2022. doi: 10.1145/3491102.3517485 2
  14. W. S. Cleveland, M. E. McGill, and R. McGill. The Shape Parameter of a Two-Variable Graph. Journal of the American Statistical Association, 83(402):289-300, 1988. doi: 10.1080/01621459.1988.10478598 9
  15. Z. Cui, S. K. Badam, M. A. Yalçin, and N. Elmqvist. DataSite: Proactive Visual Data Exploration with Computation of Insight-Based Recommen- dations. Information Visualization, 18(2):251-267, 2019. doi: 10.1177/ 1473871618806555 2
  16. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. In Pro- ceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo- gies, Volume 1 (Long and Short Papers), pp. 4171-4186. ACL, Strouds- burg, USA, 2019. doi: 10.18653/v1/N19-1423 2, 6
  17. D. H. Douglas and T. K. Peucker. Algorithms for the Reduction of the Number of Points Required to Represent a Digitized Line or Its Carica- ture. Cartographica: The International Journal for Geographic Infor- mation and Geovisualization, 10(2):112-122, 1973. doi: 10.3138/FM57 -6770-U75U-7727 5
  18. L. Earnest. The First Three Spelling Checkers. Retrieved July 18, 2023 from https://web.archive.org/web/20121022091418/http://www.stanford.edu/~l 2011. 2
  19. S. Elzer, S. Carberry, D. Chester, S. Demir, N. Green, I. Zukerman, and K. Trnka. Exploring and Exploiting the Limited Utility of Captions in Recognizing Intention in Information Graphics. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguis- tics, pp. 223-230. ACL, Stroudsburg, USA, 2005. doi: 10.3115/1219840 .1219868 2
  20. S. Elzer, S. Carberry, and I. Zukerman. The Automated Understanding of Simple Bar Charts. Artificial Intelligence, 175(2):526-555, 2011. doi: 10.1016/j.artint.2010.10.003 2
  21. M. Fasciano and G. Lapalme. PostGraphe: A System for the Generation of Statistical Graphics and Text. In Eighth International Natural Lan- guage Generation Workshop. ACL, Stroudsburg, USA, 1996. 2
  22. J. R. Finkel, T. Grenager, and C. Manning. Incorporating Non-Local In- formation into Information Extraction Systems by Gibbs Sampling. In Proceedings of the 43rd Annual Meeting of the Association for Computa- tional Linguistics, pp. 363-370. ACL, Stroudsburg, USA, 2005. doi: 10. 3115/1219840.1219885 2, 5, 9
  23. J. Fulda, M. Brehmer, and T. Munzner. TimeLineCurator: Interactive Au- thoring of Visual Timelines from Unstructured Text. IEEE Transactions on Visualization and Computer Graphics, 22(1):300-309, 2016. doi: 10. 1109/TVCG.2015.2467531 2
  24. P. Goffin, W. Willett, J.-D. Fekete, and P. Isenberg. Design Considerations for Enhancing Word-Scale Visualizations with Interaction. In Posters of the Conference on Information Visualization (InfoVis). IEEE, New York, USA, 2015. 2
  25. Google. Check Your Spelling & Grammar in Google Docs. Retrieved July 18, 2023 from https://support.google.com/docs/answer/57859?hl=en, 2023. 1, 2
  26. Google. Fix Spelling & Grammar as You Type in Gmail. Retrieved July 18, 2023 from https://support.google.com/mail/answer/7987?hl=en, 2023.
  27. Google. Use Smart Compose. Retrieved July 18, 2023 from https://support.google.com/mail/answer/9116836?hl=en, 2023. 2
  28. Google Meet. Retrieved July 18, 2023 from https://meet.google.com/, 2023. 7
  29. J. D. Gould. Looking at Pictures. In R. A. Monty and J. W. Senders, eds., Eye Movements and Psychological Processes, chap. 5.2, pp. 323- 345. Lawrence Erlbaum Associates, Inc., Hillsdale, USA, 1976. 2
  30. Grammarly. Retrieved July 18, 2023 from https://www.grammarly.com, 2023. 1, 2
  31. J. Hullman, N. Diakopoulos, and E. Adar. Contextifier: Automatic Gen- eration of Annotated Stock Visualizations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2707-2716.
  32. ACM, New York, USA, 2013. doi: 10.1145/2470654.2481374 2, 7, 9
  33. U. Jugel, Z. Jerzak, G. Hackenbroich, and V. Markl. M4: A Visualization- Oriented Time Series Data Aggregation. Proceedings of the VLDB En- dowment, 7(10):797-808, 2014. doi: 10.14778/2732951.2732953 5
  34. E. Keogh, K. Chakrabarti, M. Pazzani, and S. Mehrotra. Dimensional- ity Reduction for Fast Similarity Search in Large Time Series Databases. Knowledge and Information Systems, 3(3):263-286, 2001. doi: 10.1007/ PL00011669 5
  35. A. Khan, S. Breslav, M. Glueck, and K. Hornbaek. Benefits of Visual- ization in the Mammography Problem. International Journal of Human- Computer Studies, 83:94-113, 2015. doi: 10.1016/j.ijhcs.2015.07.001 2
  36. D. H. Kim, E. Hoque, J. Kim, and M. Agrawala. Facilitating Document Reading by Linking Text and Tables. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, pp. 423- 434. ACM, New York, USA, 2018. doi: 10.1145/3242587.3242617 2, 9
  37. D. H. Kim, V. Setlur, and M. Agrawala. Towards Understanding How Readers Integrate Charts and Captions: A Case Study with Line Charts. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 610. ACM, New York, USA, 2021. doi: 10.1145/ 3411764.3445443 1, 2, 3, 7, 9
  38. H.-K. Kong, Z. Liu, and K. Karahalios. Frames and Slants in Titles of Visualizations on Controversial Topics. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1-12. ACM, New York, USA, 2018. doi: 10.1145/3173574.3174012 2
  39. H.-K. Kong, Z. Liu, and K. Karahalios. Trust and Recall of Informa- tion across Varying Degrees of Title-Visualization Misalignment. In Pro- ceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1-13. ACM, New York, USA, 2019. doi: 10.1145/3290605. 3300576 1, 2
  40. N. Kong, M. A. Hearst, and M. Agrawala. Extracting References between Text and Charts via Crowdsourcing. In Proceedings of the SIGCHI Con- ference on Human Factors in Computing Systems, pp. 31-40. ACM, New York, USA, 2014. doi: 10.1145/2556288.2557241 2
  41. C. Lai, Z. Lin, R. Jiang, Y. Han, C. Liu, and X. Yuan. Automatic Anno- tation Synchronizing with Textual Description for Visualization. In Pro- ceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1-13. ACM, New York, USA, 2020. doi: 10.1145/3313831. 3376443 2
  42. LanguageTool. Retrieved July 18, 2023 from https://languagetool.org/, 2023. 1, 2
  43. S. Latif, D. Liu, and F. Beck. Exploring Interactive Linking between Text and Visualization. In Proceedings of the 2018 Eurographics/IEEE VGTC Conference on Visualization: Short Papers, pp. 91-94. The EG Association, Eindhoven, Netherlands, 2018. doi: 10.2312/eurovisshort. 20181084 2
  44. S. Latif, Z. Zhou, Y. Kim, F. Beck, and N. W. Kim. Kori: Interactive Synthesis of Text and Charts in Data Documents. IEEE Transactions on Visualization and Computer Graphics, 28(1):184-194, 2022. doi: 10. 1109/TVCG.2021.3114802 2
  45. LibreOffice. Checking Spelling and Gram- mar. Retrieved July 18, 2023 from https://help.libreoffice.org/6.2/en-US/text/swriter/guide/spellcheck_dialog.html, 2023. 2
  46. A. Y. Lin, J. Ford, E. Adar, and B. Hecht. VizByWiki: Mining Data Visualizations from the Web to Enrich News Articles. In Proceedings of the 2018 World Wide Web Conference, pp. 873-882. IW3C2, Geneva, Switzerland, 2018. doi: 10.1145/3178876.3186135 2
  47. A. Lundgard and A. Satyanarayan. Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content. IEEE Transactions on Visualization and Computer Graphics, 28(1):1073-1083, 2022. doi: 10.1109/TVCG.2021.3114770 2, 3, 9
  48. C. Manning, M. Surdeanu, J. Bauer, J. Finkel, S. Bethard, and D. Mc- Closky. The Stanford CoreNLP Natural Language Processing Toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computa- tional Linguistics: System Demonstrations, pp. 55-60. ACL, Stroudsburg, USA, 2014. doi: 10.3115/v1/P14-5010 2, 5, 6, 9
  49. Merriam-Webster. Retrieved July 18, 2023 from https://www.merriam-webster.com, 2023. 6
  50. L. Micallef, P. Dragicevic, and J.-D. Fekete. Assessing the Effect of Visu- alizations on Bayesian Reasoning through Crowdsourcing. IEEE Trans- actions on Visualization and Computer Graphics, 18(12):2536-2545, 2012. doi: 10.1109/TVCG.2012.199 2
  51. Microsoft. Editor Settings in Outlook.com and Out- look on the Web. Retrieved July 18, 2023 from https://support.microsoft.com/en-us/office/editor-settings-in-outl 2023. 2
  52. R. Mitkov. Anaphora Resolution. Taylor & Francis Group, London, UK, 2014. doi: 10.4324/9781315840086 9
  53. J. Obeid and E. Hoque. Chart-to-Text: Generating Natural Language De- scriptions for Charts by Adapting the Transformer Model. In Proceedings of the 13th International Conference on Natural Language Generation, pp. 138-147. ACL, Stroudsburg, USA, 2020. 2
  54. A. Ottley, A. Kaszowska, R. J. Crouser, and E. M. Peck. The Curious Case of Combining Text and Visualization. In Proceedings of the 2019 Eurographics/IEEE VGTC Conference on Visualization: Short Papers. The EG Association, Eindhoven, Netherlands, 2019. doi: 10.2312/evs. 20191181 2
  55. A. Ottley, B. Metevier, P. Han, and R. Chang. Visually Communicating Bayesian Statistics to Laypersons. In Technical Report 2012-02. Tufts University, Medford, USA, 2012. 2
  56. A. Ottley, E. M. Peck, L. T. Harrison, D. Afergan, C. Ziemkiewicz, H. A. Taylor, P. K. J. Han, and R. Chang. Improving Bayesian Reasoning: The Effects of Phrasing, Visualization, and Spatial Ability. IEEE Transactions on Visualization and Computer Graphics, 22(1):529-538, 2016. doi: 10. 1109/TVCG.2015.2467758 2 [57] Pew Research Center. Retrieved June 18, 2023 from https://www.pewresearch.org/, 2023. 3
  57. J. Pinheiro and J. Poco. ChartText: Linking Text with Charts in Docu- ments. ArXiv, abs/2201.05043, 2022. doi: 10.48550/arXiv.2201.05043 2
  58. J. Poco and J. Heer. Reverse-Engineering Visualizations: Recover- ing Visual Encodings from Chart Images. Computer Graphics Forum, 36(3):353-363, 2017. doi: 10.1111/cgf.13193 3
  59. X. Qian, E. Koh, F. Du, S. Kim, J. Chan, R. A. Rossi, S. Malik, and T. Y. Lee. Generating Accurate Caption Units for Figure Captioning. In Proceedings of the Web Conference 2021, pp. 2792-2804. ACM, New York, USA, 2021. doi: 10.1145/3442381.3449923 2
  60. U. Ramer. An Iterative Procedure for the Polygonal Approximation of Plane Curves. Computer Graphics and Image Processing, 1(3):244-256, 1972. doi: 10.1016/S0146-664X(72)80017-0 5
  61. K. Rong and P. Bailis. ASAP: Prioritizing Attention via Time Series Smoothing. Proceedings of the VLDB Endowment, 10(11):1358-1369, 2017. doi: 10.14778/3137628.3137645 5
  62. P. Rosen and G. J. Quadri. LineSmooth: An Analytical Framework for Evaluating the Effectiveness of Smoothing Techniques on Line Charts. IEEE Transactions on Visualization and Computer Graphics, 27(2):1536- 1546, 2021. doi: 10.1109/TVCG.2020.3030421 5
  63. C. Stokes, V. Setlur, B. Cogley, A. Satyanarayan, and M. A. Hearst. Strik- ing a Balance: Reader Takeaways and Preferences when Integrating Text and Charts. IEEE Transactions on Visualization and Computer Graphics, 29(1):1233-1243, 2023. doi: 10.1109/TVCG.2022.3209383 2
  64. J. Sweller, P. Ayres, and S. Kalyuga. The Split-Attention Effect. In Cogni- tive Load Theory, chap. 9, pp. 111-128. Springer, New York, USA, 2011. doi: 10.1007/978-1-4419-8126-4_9 2
  65. Tableau Public. Retrieved July 18, 2023 from https://public.tableau.com/app/discover, 2023. 1, 3 [68] Tableau Software. Retrieved July 18, 2023 from https://www.tableau.com, 2023. 1, 2, 3
  66. The New York Times. Retrieved July 18, 2023 from https://www.nytimes.com/, 2023. 3
  67. E. R. Tufte. The Visual Display of Quantitative Information. Graphics Press, Cheshire, USA, 2001. 2
  68. U.S. Department of the Treasury. Retrieved July 18, 2023 from https://home.treasury.gov/, 2023. 3
  69. Vox Media. Retrieved July 18, 2023 from https://www.vox.com/, 2023. 3
  70. M. P. Whitacre and E. W. Saul. High School Girls' Interpretations of Science Graphs: Exploring Complex Visual and Natural Language Hy- brid Text. International Journal of Science and Mathematics Education, 14(8):1387-1406, 2016. doi: 10.1007/s10763-015-9677-7 2 [76] Wikimedia Commons. Retrieved July 18, 2023 from https://commons.wikimedia.org/wiki/Main_Page, 2023. 2
  71. G. Wills and L. Wilkinson. AutoVis: Automatic Visualization. Informa- tion Visualization, 9(1):47-69, 2010. doi: 10.1057/ivs.2008.27 2 [78] WolframAlpha. Retrieved July 18, 2023 from https://www.wolframalpha.com/, 2023. 9
  72. C. Xiong, L. Van Weelden, and S. Franconeri. The Curse of Knowledge in Visual Data Communication. IEEE Transactions on Visualization and Computer Graphics, 26(10):3051-3062, 2020. doi: 10.1109/TVCG.2019 .2917689 1, 2, 3
  73. H. H. Zhu, S. S. Cheng, and E. Wu. How Do Captions Affect Visualiza- tion Reading? ArXiv, abs/2205.01263, 2022. doi: 10.48550/arXiv.2205. 01263 1, 2
  74. Çagatay Demiralp, P. J. Haas, S. Parthasarathy, and T. Pedapati. Foresight: Rapid Data Exploration Through Guideposts. ArXiv, abs/1709.10513, 2017. doi: 10.48550/arXiv.1709.10513 2

Related papers

Towards Understanding How Readers Integrate Charts and Captions: A Case Study with Line Charts
Vidya Setlur

Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021

Charts often contain visually prominent features that draw attention to aspects of the data and include text captions that emphasize aspects of the data. Through a crowdsourced study, we explore how readers gather takeaways when considering charts and captions together. We frst ask participants to mark visually prominent regions in a set of line charts. We then generate text captions based on the prominent features and ask participants to report their takeaways after observing chart-caption pairs. We fnd that when both the chart and caption describe a high-prominence feature, readers treat the doubly emphasized high-prominence feature as the takeaway; when the caption describes a low-prominence chart feature, readers rely on the chart and report a higher-prominence feature as the takeaway. We also fnd that external information that provides context, helps further convey the caption's message to the reader. We use these fndings to provide guidelines for authoring efective chart-caption pairs. CCS CONCEPTS • Human-centered computing → Empirical studies in visualization.

View PDFchevron_right
Striking a Balance: Reader Takeaways and Preferences when Integrating Text and Charts
Bridget Cogley

IEEE Transactions on Visualization and Computer Graphics

Fig. 1. Information displays with varying amounts of visuals and text. (a) Chart presented with no text (beyond axes and ticks), (b) Chart with a title and a single annotation, (c) Chart which displays a narrative or story around the data, annotated through text, and (d) A text-only version of the data, with the same story as displayed in (c).

View PDFchevron_right
Pay "Attention" to Chart Images for What You Read on Text
Xiaoman Zhao

Companion of the 2023 International Conference on Management of Data

Data visualization is changing how we understand data, by showing why's, how's, and what's behind important patterns/trends in almost every corner of the world, such as in academic papers, news articles, financial reports, etc. However, along with the increasing complexity and richness of data visualization, given a text description (e.g., "fewer teens say they attended school completely online (8%)"), it becomes harder for users to pinpoint where to pay attention to on a chart (e.g., a grouped bar chart). In this demonstration paper, we present a system HiChart for text-chart image highlighting: when a user selects a span of text, HiChart automatically analyzes the chart image (e.g., a jpeg or a png file) and highlights the parts that are relevant to the span. From a technical perspective, HiChart devises the following techniques. Reverse-engineering visualizations: given a chart image, HiChart uses computer vision techniques to generate a visualization specification using Vega-Lite language, as well as the underlying dataset; Visualization calibration by data tuning: HiChart calibrates the regenerated chart by tuning the recovered dataset through value perturbation; and Chart highlighting for a span: HiChart maps the span to corresponding data cells and uses the built-in highlighting functions of Vega-Lite to highlight the chart. CCS CONCEPTS • Human-centered computing → Visualization.

View PDFchevron_right
The Curious Case of Combining Text and Visualization
R. Crouser

2019

Visualization research has made significant progress in demonstrating the value of graphical data representation. Even still, the value added by static visualization is disputed in some areas. When presenting Bayesian reasoning information, for example, some studies suggest that combining text and visualizations could have an interactive effect. In this paper, we use eye tracking to compare how people extract information from text and visualization. Using a Bayesian reasoning problem as a test bed, we provide evidence that a visualization makes it easier to identify critical information, but that once identified as critical, information is more easily extracted from text. These tendencies persist even when text and visualization are presented together, indicating that users do not integrate information well across the two representation types. We discuss these findings and argue that effective representations should consider the ease of both information identification and extraction...

View PDFchevron_right
ChartText: Linking Text with Charts in Documents
Jorge L Poco

2022

Recent works show that interactive documents connecting text with visualizations facilitate reading comprehension. However, creating this type of content requires specialized knowledge. We present ChartText, a method that links text with visualizations in this work. Our approach supports documents that include bar charts, line charts, and scatter plots. ChartText receives the visual encoding of the visualization and its associated text as input. It then performs the linking in two stages: The matching stage creates individual links relating simple phrases between the text and the chart. Then, it combines the individual links according to the visual channels in the grouping stage, building more meaningful connections. We use two datasets to design and evaluate our method; the first comes from web documents (24 bar charts and texts) and the second from academic documents (25 bar charts, 25 line charts, and 25 scatter plots with their texts). Our experiments show that our method obtain...

View PDFchevron_right

Related topics

  • Computer Science
  • Information Retrieval
  • Pie Chart
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved