Towards retrieving relevant information graphics

2018

Images greatly help in understanding, interpreting and visualizing data. Adding textual description to images is the first and foremost principle of web accessibility. Visually impaired users using screen readers will use these textual descriptions to get better understanding of images present in digital contents. In this paper, we propose Chart-Text a novel fully automated system that creates textual description of chart images. Given a PNG image of a chart, our Chart-Text system creates a complete textual description of it. First, the system classifies the type of chart and then it detects and classifies the labels and texts in the charts. Finally, it uses specific image processing algorithms to extract relevant information from the chart images. Our proposed system achieves an accuracy of 99.72% in classifying the charts and an accuracy of 78.9% in extracting the data and creating the corresponding textual description.

2021 IEEE 14th Pacific Visualization Symposium (PacificVis)

Figure 1: Infographics effectively convey messages using a mix of text and pictures (a). Automatically parsing infographics is challenging due to abstract concepts and diverse visual styles. We present an approach to detect and tag icons in infographics (see Fig. 8 for final results). We accomplish this by curating and annotating a dataset of infographics called Visually29K (b), and training an icon detection model on synthetic data generated by pasting icons onto patches of infographics (c). We also present three demo applications: topic prediction, multi-modal summarization-outputing text tags and visual hashtags representative of an infographic's topics (d), and multi-modal search-prioritizing infographics containing both visual and textual elements matching a query. ©oceanservice.noaa.gov, Evanmade Graphic Design, Walker Sands, Overson Roofing.

2011

Poorly designed charts are prevalent in reports, magazines, books and on the Web. Most of these charts are only available as bitmap images; without access to the underlying data it is prohibitively difficult for viewers to create more effective visual representations. In response we present ReVision, a system that automatically redesigns visualizations to improve graphical perception. Given a bitmap image of a chart as input, ReVision applies computer vision and machine learning techniques to identify the chart type (eg, pie chart, ...

Lecture Notes in Computer Science, 2013

Information graphics (such as bar charts and line graphs) are widely used in popular media. The majority of such non-pictorial graphics have the purpose of communicating a high-level message which is often not repeated in the text of the article. Thus, information graphics together with the textual segments contribute to the overall purpose of an article and cannot be ignored. Unfortunately, information graphics often do not label the dependent axis with a full descriptor of what is being measured. In order to realize the high-level message of an information graphic in natural language, a referring expression for the dependent axis must be generated. This task is complex in that the required referring expression often must be constructed by extracting and melding pieces of information from the textual content of the graphic. Our heuristic-based solution to this problem has been shown to produce reasonable text for simple bar charts. This paper presents the extensibility of that approach to other kinds of graphics, in particular to grouped bar charts and line graphs. We discuss the set of component texts contained in these two kinds of graphics, how the methodology for simple bar charts can be extended to these kinds, and the evaluation of the enhanced approach.

Lecture Notes in Computer Science, 2013

Information graphics (bar charts, line graphs, etc.) in popular media generally have a high-level message that they are intended to convey. These messages are seldom repeated in the document's text yet contribute to understanding the overall document. The relative perceptual effort required to recognize a particular message is a communicative signal that serves as a clue about whether that message is the one intended by the graph designer. This paper presents a model of relative effort by a viewer for recognizing different messages from grouped bar charts. The model is implemented within the ACT-R cognitive framework and has been validated by human subjects experiments. We also present a statistical analysis of the contribution of effort in recognizing the intended message of a grouped bar chart.

ACL HLT …, 2011

Information graphics (bar charts, line graphs, etc.) in popular media generally have a discourse goal that contributes to achieving the communicative intent of a multimodal document. This paper presents our work on abstractive summarization of line graphs. Our methodology involves hypothesizing the intended message of a line graph and using it as the core of a summary of the graphic. This core is then augmented with salient propositions that elaborate on the intended message.

Companion of the 2023 International Conference on Management of Data

Data visualization is changing how we understand data, by showing why's, how's, and what's behind important patterns/trends in almost every corner of the world, such as in academic papers, news articles, financial reports, etc. However, along with the increasing complexity and richness of data visualization, given a text description (e.g., "fewer teens say they attended school completely online (8%)"), it becomes harder for users to pinpoint where to pay attention to on a chart (e.g., a grouped bar chart). In this demonstration paper, we present a system HiChart for text-chart image highlighting: when a user selects a span of text, HiChart automatically analyzes the chart image (e.g., a jpeg or a png file) and highlights the parts that are relevant to the span. From a technical perspective, HiChart devises the following techniques. Reverse-engineering visualizations: given a chart image, HiChart uses computer vision techniques to generate a visualization specification using Vega-Lite language, as well as the underlying dataset; Visualization calibration by data tuning: HiChart calibrates the regenerated chart by tuning the recovered dataset through value perturbation; and Chart highlighting for a span: HiChart maps the span to corresponding data cells and uses the built-in highlighting functions of Vega-Lite to highlight the chart. CCS CONCEPTS • Human-centered computing → Visualization.

Proceedings of the Fifth International Natural Language Generation Conference on - INLG '08, 2008

Information graphics, such as bar charts and line graphs, play an important role in multimodal documents. This paper presents a novel approach to producing a brief textual summary of a simple bar chart. It outlines our approach to augmenting the core message of the graphic to produce a brief summary. Our method simultaneously constructs both the discourse and sentence structures of the textual summary using a bottom-up approach. The result is then realized in natural language. An evaluation study validates our generation methodology.

The availability of multimedia material on the Internet as well as on private Intranets and multimedia collections poses new challenges for information access systems. We present a number of experiments on identification of conceptual information in graphics which can be used for semantic indexing and search. We investigate the use of rulebased and machine learning systems which operate over clean and noisy textual sources associated to images and charts for the identification of concepts expressed in a domain ontology and which are used to index the images. Results show that the investigated techniques support the process of image annotation.

Information

Knowing how to construct text-based Search Queries (SQs) for use in Search Engines (SEs) such as Google or Wikipedia has become a fundamental skill. Though much data are available through such SEs, most structured datasets live outside their scope. Visualization tools aid in this limitation, but no such tools come close to the sheer amount of information available through general-purpose SEs. To fill this gap, this paper presents Q4EDA, a novel framework that converts users’ visual selection queries executed on top of time series visual representations, providing valid and stable SQs to be used in general-purpose SEs and suggestions of related information. The usefulness of Q4EDA is presented and validated by users through an application linking a Gapminder’s line-chart replica with a SE populated with Wikipedia documents, showing how Q4EDA supports and enhances exploratory analysis of United Nations world indicators. Despite some limitations, Q4EDA is unique in its proposal and rep...