GraSSML: accessible smart schematic diagrams for all

2017

Conventional methods for describing complex diagrams to blind people are either ineffective or inefficient. We describe our preliminary work on how one can describe a diagram to a blind person with minimal supervision. We explain our text localization method which beats commercially available off-the-shelf state-of-the-art systems. We also provide a prototype user interface which can effectively describe diagrams based on our user study.

2016

The corner stone of a language development environment is an editor. For programming languages, several code editors are readily available to be integrated in Web applications. However, only few editors exist for diagrammatic languages. Eshu is an extensible diagram editor, embeddable in Web applications that require diagram interaction, such as modeling tools or e-learning environments. Eshu is a JavaScript library with an API that supports its integration with other components, including importing/exporting diagrams in JSON. Eshu was already integrated in a pedagogical environment with automated diagram assessment, configured for extended entity-relationship diagrams, that served as basis for an usability evaluation.

2017

A classic Integrated Development Environment (IDE) allows displaying information only with a textual representation. This kind of representation is perfect for the linear aspect of the code, but not effective to represent links between code fragments. Current graphical code representation modules in IDE are suited to apprehend the system from a global point of view. However, the cognitive integration cost of those diagrams is disproportionate related to the elementary coding task. Our approach considers graphical representation but only with code elements that are parts of the developer's mental model during his programming task. The corresponding cognitive integration of our graphical representation is then less costly and the information that text struggles to display will be clearly explicit. We use UML for this representation because it is a widespread and well-known formalism. We want to show that dynamic diagrams, whose content is modified and adapted in real-time by monit...

Proceedings of the …, 2005

Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A), 2010

Information graphics (such as bar charts and line graphs) play a vital role in many multimodal documents. Unfortunately, visually impaired individuals who use screen reader programs to navigate through such documents have limited access to the graphics. This paper presents the Interactive SIGHT (Summarizing Information GrapHics Textually) system that provides visually impaired individuals with the high-level knowledge that one would gain from viewing graphics in electronic documents. The current system, which is implemented as a browser extension, works on simple bar charts. Once launched by a keystroke combination, Interactive SIGHT first provides a brief initial summary that conveys the underlying message of the bar chart along with the chart's most significant features. The system is then able to generate history-aware follow-up responses that provide further information upon request from the user. User evaluations with sighted and visually impaired users showed that the initial summary and follow-up responses are very effective in conveying the informational content of graphics and that the system interface is easy to use.

Proceedings of the 11th annual ACM symposium on User interface software and technology, 1998

The intelligent diagram is a recent metaphor for diagramming in which the underlying graphic editor parses the diagram as it is being constructed, performing error correction and collecting geometric constraints which capture the relationships between diagram components. During diagram manipulation a constraint solver uses these geometric constraints to maintain the diagram's semantics. We describe the Penguins system. This automates the development of graphical editors that support the intelligent diagram metaphor. It takes a grammatical specification of a particular diagram language and generates an editor that supports the creation, manipulation and parsing of diagrams in that visual language. Our empirical results show that the system can be used to generate specialized editors for a wide variety of diagram languages, ranging from state transition diagrams to mathematical equations, with real-time incremental parsing, error correction and direct manipulation.

2004

One aspect of non-visual visualisation is providing accessibility to diagrams for users with visual impairments. In this paper we review the literature on diagrams and nonvisual presentation in order to elucidate the issues involved in making graph based diagrams accessible using speech and non-speech sound. First we examine the nature, scope and uses of these diagrams. We then describe the nature of diagrams: how do diagrams differ from other representations; how do sighted readers read, understand, and extract information from diagrams; what cognitive processes do diagrams facilitate; and what factors affect how diagrams may be understood? After a comparison of visual with aural presentation we discuss the work presented by others in this field, particularly looking at their reasons for implementing how they do in light of our examination of visual comprehension. The paper concludes with a discussion of how these issues combine and conflict to influence requirements for interface design.

ACM Transactions on Computer-Human Interaction, 2003

The intelligent diagram is a recent metaphor for diagramming in which the underlying graphic editor parses the diagram as it is being constructed, performing error correction and collecting geometric constraints that capture the relationships between diagram components. During diagram manipulation a constraint solver uses these geometric constraints to maintain the diagram's semantics. We introduce the Penguins system. This automates the development of graphical editors that support the intelligent diagram metaphor. It takes a grammatical specification of a particular diagram language and generates an editor specialized for the creation, manipulation and parsing of diagrams in that visual language. We extend previous research in this area by allowing more expressive grammars, performing automatic error correction, and detailing how efficient incremental parsing has been achieved. We also provide an empirical evaluation of the system. This shows that the system can be used to gen...

2008

Graphical representations are a powerful way of conveying information. Their use has made life much easier for most sighted users, but people with disabilities or users who work in environments where visual representations are inappropriate cannot access information contained in graphics, unless alternative descriptions are included. We describe an approach called Graphical Structure Semantic Markup Languages (GraSSML) which aims at defining high-level diagram description languages which capture the structure and the semantics of a diagram and enable the generation of accessible and “smart ” presentations in different modalities such as speech, text, graphic, etc. The structure and the semantics of the diagram are made available at the creation stage. This offers new possibilities for allowing Web Graphics to become “smart”.

Universal Access in the Information Society, 2014

One of the major barriers for the social inclusion of blind persons is the limited access to graphics-based learning resources which are highly vision oriented. This paper presents a cost-effective tool which facilitates comprehension and creation of virtual directed graphs, such as flowcharts, using alternate modalities of audio and touch. It provides a physically accessible virtual spatial workspace and multimodal interface to non-visually represent directed graphs in interactive manner. The concept of spatial query is used to aid exploration and mental visualization through audio and tactile feedback. A unique aspect of the tool, named DiGVis, offers compatible representations of directed graphs for the sighted and non-sighted persons. A study with 28 visually challenged subjects indicates that the tool facilitates comprehension of layout and directional connectivity of elements in a virtual diagram. Further, in a pilot study, blind persons could independently comprehend a virtual flowchart layout and its logical steps. They were also able to create the flowchart data without sighted assistance using DiGVis. A comparison with sighted subjects using DiGVis for similar task demonstrates the effectiveness of the technique for inclusive education.