Improving test software using TTCN-3

Proceedings of the 2015 International Symposium on Software Testing and Analysis, 2015

Writing a test harness is a difficult and repetitive programming task, and the lack of tool support for customized automated testing is an obstacle to the adoption of more sophisticated testing in industry. This paper presents TSTL, the Template Scripting Testing Language, which allows users to specify the general form of valid tests for a system in a simple but expressive language, and tools to support testing based on a TSTL definition. TSTL is a minimalist templatebased domain-specific language, using the source language of the Software Under Test (SUT) to support most operations, but adding declarative idioms for testing. TSTL compiles to a common testing interface that hides the details of the SUT and provides support for logging, code coverage, delta debugging, and other core testing functionality, making it easy to write universal testing tools such as random testers or model checkers that apply to all TSTL-defined harnesses. TSTL is currently available for Python, but easily adapted to other languages as well.

IEEE Software, 2000

Software developers face severe testing challenges today in terms of increasing system complexity, reduced time to market, and stringent quality requirements. These challenges are particularly prevalent in mobile telecommunications system development. TTCN-3 is a standardized testing language that offers a possible solution to many current testing challenges. However, introducing a new technology in a large organization is a complex process in itself. The authors describe their experience addressing these complexities with TTCN-3 at Nokia.This article is part of a special issue on Software Testing.Deiss, T.; Nyberg, A.J.; Schulz, S.; Teittinen, R.; Willcock, C.Software, IEEE48-54July-Aug. 2006学术搜索 www.libsou.com scholar search 4 8 I E E E S O F T W A R E P u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y 0 7 4 0 -7 4 5 9 / 0 6 / $ 2 0 . 0 0 © 2 0 0 6 I E E E

Testing of Communicating Systems XIV, 2002

This paper discusses the implementation of test systems, which are specified based on TTCN version 3 (TTCN-3). TTCN-3 has recently been standardized by [4] European Telecommunications Standards Institute (ETSI) as a new test specification language beyond protocol testing. TTCN-3 source code however only constitutes the first step in a TTCN-3 test system implementation. A complete test system implementation also requires the specification of a test and platform specific adaptation layer. This issue is addressed in the TTCN-3 Runtime Interface (TRI) draft standard also developed at ETSI. The TRI is expected to become the future standard interface for all TTCN-3 test system implementations. We present different kinds of test systems which can be built from the conceptual model provided by the TRI. Next to a discussion on the application of these concepts for classic conformance testing, we also propose a new approach, which significantly simplifies the specification and implementation of TTCN test systems for IP-based applications. We illustrate this approach with the presentation of an application example.

2001

Recently, international standard bodies approved the third evolution of the TTCN test specification language (TTCN-3) as a requirement to modernise and widen its application beyond pure OSI conformance testing. Even though TTCN-3 makes the description of complex distributed test behaviour much easier there is still a requirement from the user community to provide a visualisation means for test specification, development and tracing. Message Sequence Charts (MSC) appeared to be a particularly attractive candidate as a graphical means for visualising TTCN-3 test cases. Therefore, in addition to the pure textual core language, TTCN-3 also defines a MSC profile called the Graphical Presentation Format for TTCN (GPF). This paper describes some of the main concepts of GPF, its use and application to real test suite examples.

1996

Although internationally standardized (or are in the process of becoming international standard), the definition of TTCN and concurrent TTCN can still be improved. To solve some ofthe known problems we propose a test specification language, called PROSPECT -PROtocol test SPEcification language for Conformance Testing. The proposed language has a formally defined syntax and has a formally defined semantics. We have introduced test modules in our language so that the distribution of test components over real systems can be specified. As we assume that in every test module a main test component exists responsible for creating and terminating other test components, we have been in a position to give our language a precise semantics. In order to prove PROSPECT's suitability we have implemented a compiler and a runtime library. The implementation has been done in a UNIX™ environment. The details of our approach are described in this paper.

2000

The Tree and Tabular Combined Notation (TTCN) is a semi-formal notation which supports the specification of abstract test suites for protocol conformance testing. An abstract test suite is a collection of abstract test cases 1) . As indicated by the name TTCN, test cases are described in the form of behavior trees and different kinds of tables are used for the graphical representation of test suites. Ina Schieferdecker (33) received her PhD from the Technical University Berlin in 1994. Since 1993 she has been a researcher at GMD Fokus, and a lecturer at Technical University Berlin since 1995. Her interests cover testing methods for distributed systems and formal methods for the design, validation and prototyping of distributed systems. She has been head of the Competence Center for Testing, Interoperability and Performance (TIP) since 1997 and is actively involved in several testing projects. She has published several papers on testing telecommunications systems and developing test systems, and is involved in the definition of MSC in ITU-T SG10 and of TTCN-3 in ETSI. Schieferdecker@fokus.gmd.de Jens Grabowski (38) graduated from the University of Hamburg with a diploma degree in Computer Science and Chemistry. He spent two years at SIEMENS AG in Munich focusing on protocol specification and protocol validation based on Petri Nets, SDL and MSC. 1990-1995 he was research scientist at the University of Berne, where he received his PhD in 1994. Since 1995 Grabowski has been researcher and lecturer at the Institute for Telematics at the Medical University in Lübeck; since 1996 he has also worked as expert in several ETSI standardization projects. He is a member of the ETSI experts team which develops the third edition of TTCN. jens@itm.mu-luebeck.de 1) CTMF and TTCN use the terms abstract and executable to distinguish between implementationindependent and implementation-dependent concepts, e.g. abstract test suite and executable test suite, abstract test case and executable test case or abstract service primitive. This paper introduces mainly implementation-independent CTMF and TTCN concepts. Qualifiers like abstract or executable will only be used in case of ambiguities.

ACM Standardview, 1996

Ⅵ The increase in distributed applications and the need for information-sharing has led to increased demand for information-exchange standards to define the means by which applications can communicate. Although testing is critical to ensuring interoperable products, it does add time and cost to the development process. This article reviews work in progress on testing for the STEP standard that promises to reduce the cost of developing standards and standards-based products. An investigation of common approaches to test suite development and testing methods leads to some insights on ways to improve the overall process. Development time and cost can actually be reduced by better applying testing methods and tools.

International Journal on Software Tools for Technology Transfer, 2013

With more than 10 years of maturing through industrial use and standardization, the Testing and Test Control Notation (TTCN-3) has become a widely used technology that many businesses depend upon for ensuring their product quality. With the rising number of supporters and tools, the demand for a means to assess the standards compliance of TTCN-3 tools has increased. In this article, we describe the motivation, approach, methodology, and results of the still ongoing project to develop a standardized conformance test suite for TTCN-3 tools. We discuss the challenges involved in creating such a test suite, the way to deal with imposed resource limitations of the project, and where we think the effort is heading.

As the size and complexity of large software systems continuously grow, so do their test systems. In today's telecommunication world, we have test systems which are comparable in complexity to that of the tested systems. Some of these test systems have to simulate millions of users, be as robust as the tested systems themselves and provide comparable performance. To be able to handle such testing needs, ETSI developed the TTCN-3 programming language. TTCN-3 has proved to be very efficient for developing test systems for communication systems. In this article we present the results of analyzing the test software systems publicly available from www.ttcn-3.org. We show the issues that we have found on semantic level and on advanced code smell level. We also present some results on the test systems structural level.

IFIP Advances in Information and Communication Technology, 1999

Writing tests for concurrent systems is difficult and time consuming. Handling concurrency requires an expressive language, yet the same language should be easy to read and edit. A standardized test description language such as TTCN is often perceived as too cryptic, difficult to read and generally too powerful. Other test description languages are home made and do not allow to share or to maintain easily existing test suites. We present a test specification and execution tool relying on Msc. The Msc language is standardized, and provides a graphical and intuitive way for describing test cases. ' work supported by the CS Communications & Systems Group J. Wu et al. (eds.