Patterns for Evaluating Usability of Domain-Specific Languages

Journal of Systems and Software, 2015

A domain-specific language (DSL) aims to support software development by offering abstractions pertinent to a particular software domain. It is expected that DSLs improve the maintainability of artifacts otherwise produced with general-purpose languages. However, the maintainability of the DSL artifacts and, hence, their adoption in mainstream software development, is largely dependent on the usability of the language. More than a single DSL is often proposed to address the same software engineering problem. Unfortunately, it is also often hard to identify their usability strengths and weaknesses early, as there is no guidance on how to objectively reveal them. Usability is a multi-faceted quality characteristic, which is challenging to quantify before a DSL is actually used by its stakeholders. There is even less support and experience on how to quantitatively evaluate the usability of DSLs used in software maintenance tasks. In this context, this paper reports a study to compare the usability of textual DSLs under the perspective of software maintenance. A usability measurement framework was developed based on the Cognitive Dimensions of Notations (CDN). The framework was evaluated both qualitatively and quantitatively using two textual DSLs in the context of two evolving object-oriented systems. The results suggested that the proposed metrics were useful: (1) to early identify the DSL usability limitations to be addressed, (2) to reveal specific features of the DSLs favoring software maintenance tasks, and (3) to successfully analyze eight usability dimensions that are critical in many DSLs.

ACM SIGAPP Applied Computing Review, 2018

Software engineers are increasingly taking advantage of new methods to improve software quality. The use of languages developed for specific domains, which in the literature are known as Domain-Specific Languages (DSLs), has grown in the past years. Although several experimental studies that subjectively evaluate usability of these languages can be found in the literature, few of them have taken advantage of applying Human-Computer Interaction (HCI) techniques in those evaluations. Therefore, the main goals of this paper are to present a usability evaluation framework for DSLs, called Usa-DSL, and to show the evaluation of the framework through a Focus Group method. The evaluation was performed by seven specialists that discussed the framework usability and suggested some modifications of our initial proposal. The specialists recommendations were incorporated in the final framework presented in this paper.

Domain-Specific Languages (DSLs) are claimed to increment productivity, while reducing the required maintenance and programming expertise. In this context, DSLs usability is a key factor for its successful adoption. In this paper, we propose a systematic approach based on User Interfaces Experimental validation techniques to assess the impact of the introduction of DSLs on the productivity of domain experts. To illustrate this evaluation approach we present a case study of a DSL for High Energy Physics (HEP). The DSL on this case study, called Pheasant (PHysicist's EAsy Analysis Tool), is assessed in contrast with a pre-existing baseline, using General Purpose Languages (GPLs) such as C++. The comparison combines quantitative and qualitative data, collected with users from a real-world setting. Our assessment includes Physicists with programming experience with two profiles; ones with no experience with the previous framework used in the project and other experienced. This work's contribution highlights the problem of the absence of systematic approaches for experimental validation of DSLs. It also illustrates how an experimental approach can be used in the context of a DSL evaluation during the Software Languages Engineering activity, with respect to its impact on effectiveness and efficiency.

Proceedings of the 3rd ACM SIGPLAN workshop on Evaluation and usability of programming languages and tools - PLATEAU '11, 2011

Domain-Specific Languages (DSLs) are claimed to increment productivity, while reducing the required maintenance and programming expertise. In this context, DSLs usability is a key factor for its successful adoption.

Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 2021

Usability evaluation of a Domain-Specific Language (DSL) is not a simple task, since DSL designers effort might not be viable in a project context. Hence, we ease DSL designers work by providing a fast and simple way to evaluate their languages and, therefore, reduce effort (spend time) when a new DSL is developed. In order to do that, this paper presents a structured way to build a Heuristic Evaluation Checklist (HEC) for DSLs. This checklist is different from traditional checklists since it is focused on DSL. Once a checklist is provided, the evaluators only follow a set of heuristics and freely point out the found errors when using the DSL. Basically, the produced checklist provides a set of questions, based on the heuristics that direct an evaluation for a specific domain. In order to show how our proposal can be applied to a DSL and to provide an initial evaluation of our HEC, this paper shows also an instance to evaluate graphical and textual DSLs. Furthermore, this paper also discusses the qualitative analysis of an initial evaluation for the proposed HEC through seven interviews with Human-Computer Interaction (HCI) experts. Finally, a brief example of use applying the developed checklist is presented.

Proc. of ECOOP Workshop on Domain-Specific Program Development (DSPD), 2006

A domain-specific language (DSL), whether used for model-driven development or programming, is a piece of critical infrastructure that is developed during the system engineering process. As such, a DSL has its own lifecycle, which in turn may encapsulate the lifecycles of many other system development projects. Understanding the requirements for DSLs in general, and in specific project contexts, is critical in order to improve DSL quality and ensure a direct correspondence between the requirements for system ...

Domain-Specific Languages (DSLs) developers aim to narrow the gap between the level of abstraction used by domain users and the one provided by the DSL, in order to help taming the increased complexity of computer systems and real-world problems. The quality in use of a DSL is essential for its successful adoption. We illustrate how a usability evaluation process can be weaved into the development process of a concrete DSL -FlowSL -used for specifying humanitarian campaign processes lead by an international Non-Governmental Organization. FlowSL is being developed following an agile process using Model-Driven Development (MDD) tools, to cope with vague and poorly understood requirements in the beginning of the development process.

Designing a new domain specific language is as any other complex task sometimes error-prone and usually time con- suming, especially if the language shall be of high-quality and comfortably usable. Existing tool support focuses on the simplification of technical aspects but lacks support for an enforcement of principles for a good language design. In this paper we investigate guidelines that are useful for de- signing domain specific languages, largely based on our ex- perience in developing languages as well as relying on ex- isting guidelines on general purpose (GPLs) and modeling languages. We defined guidelines to support a DSL devel- oper to achieve better quality of the language design and a better acceptance among its users.

Computer Science …, 2010

Many domain-specific languages, that try to bring feasible alternatives for existing solutions while simplifying programming work, have come up in recent years. Although, these little languages seem to be easy to use, there is an open issue whether they bring advantages in comparison to the application libraries, which are the most commonly used implementation approach. In this work, we present an experiment, which was carried out to compare such a domain-specific language with a comparable application library. The experiment was conducted with 36 programmers, who have answered a questionnaire on both implementation approaches. The questionnaire is more than 100 pages long. For a domain-specific language and the application library, the same problem domain has been used -construction of graphical user interfaces. In terms of a domain-specific language, XAML has been used and C# Forms for the application library. A cognitive dimension framework has been used for a comparison between XAML and C# Forms.