Papers by Jürgen Großmann
A Trace Management Platform for Risk-Based Security Testing
Lecture Notes in Computer Science, 2014
The goal of risk-based security testing is to improve the security testing process in order to co... more The goal of risk-based security testing is to improve the security testing process in order to cover especially risky areas of the application under test and at the same time minimize the time to market and to improve the use of resources by focusing testing work on areas with the highest risks. In RBST risk factors are identified and risk-based security test cases are created and prioritized according to an applicable selection strategy. One of the challenges in RBST is to keep track of the different artifacts that are often managed by different tools. Traceability is the key to manage complex systems in development and testing. This paper introduces RISKTest, a trace management platform on the basis of Eclipse that supports the creation and documentation of cross-tool relations during test development and test execution. RISKTest is dedicated to risk-based security testing. Thus, we concentrate on the management of traces between the artifacts from risk assessment and testing and the definitions of services that automatically analyze the related artifacts for security and testing related aspects. RISKTest has been developed in the DIAMONDS and RASEN projects and evaluated within the project’s case studies.
TTCN-3 has gained increasing significance in recent years. Originally developed to fit the needs ... more TTCN-3 has gained increasing significance in recent years. Originally developed to fit the needs for testing software-based applications and systems in the telecommunication industry, TTCN-3 has shown its applicability to a wide range of other industrial domains in the mean time. TTCN-3 provides platform-independent, universal and powerful concepts to describe tests -- especially for discrete, interactive systems -- on different levels of abstraction. However, TTCN-3 addresses systems with discrete input and output characteristics only. In the automotive industry -- as well as in other industries that deal with highly complex software-based control systems -- this is not sufficient. Control systems often interact with their environment trough sensors and actuators using continuous signals. A test environment that adequately supports the specification, execution and evaluation of tests for embedded control systems has to provide concepts to handle this kind of signals. Moreover it ha...
Member of the ACM
Test processes in the automotive industry are tool-intensive and affected by technologically hete... more Test processes in the automotive industry are tool-intensive and affected by technologically heterogeneous test infrastructures. In the industrial practice a product has to pass tests at several levels of abstraction such as Model-in-the-Loop (MIL), Software-in-the-Loop (SIL) and Hardware-in-the-Loop (HIL) tests. Different test systems are applied for this purpose (e.g. dSPACE MTest, dSPACE Automation Desk, National Instruments Teststand) and almost each test system requests its own proprietary test description language. The exchange of tests between different test systems and the reuse of tests between different test levels is normally not possible. Efforts to integrate these heterogeneous test environments, to address test exchange in a general manner and to standardize and harmonize the existing language environment are still at the beginning and not tailored towards the requirements of the automotive domain. To keep the whole development and test process efficient and manageable...
Data Fuzzing with TTCN-3
Fuzz testing or fuzzing is a commonly used method to test for security problems in software or co... more Fuzz testing or fuzzing is a commonly used method to test for security problems in software or computer systems. It is a black-box testing technique in which the system under test is stressed with invalid, unexpected or random data inputs and data structures through its interfaces. The purpose of fuzzing is to reveal implementation vulnerabilities by triggering failure modes. This is done by stimulating the system with unexpected data in the form of modified valid data, and observing the behaviour of the system. In our presentation we will propose a light-weight extension to the current TTCN-3 standard that supports fuzzing with TTCN-3 to maximize its usability for existing TTCN-3 users. Fuzzing operations are defined on basis of the TTCN-3 type system and formally specified by subset of the TTCN-3 template language. The fuzzing itself (i.e. the generation of fuzzed data) is done on the fly during the call of the send operation. The repeated application of data fuzzing, i.e. generat...
Testml a language for exchange of tests
Lecture Notes in Computer Science, 2008
Test evaluation and test assessment is a time consuming and resource intensive process. More than... more Test evaluation and test assessment is a time consuming and resource intensive process. More than ever this holds for testing complex systems that emanate continuous or hybrid behavior. In this article we introduce an approach that eases the specification of black box tests for hybrid or continuous systems by means of signal properties applied for evaluation. A signal property allows the characterization of individual signal shapes. It is determined by the examination of the signal's value at time, the application of pre-processing functions (like first or higher order derivatives), and the analysis and detection of sequences of values that form certain shapes of a signal (e.g. local minima and maxima). Moreover we allow the combination of properties by logical connectives. The solution provided in this paper is based on terms and concepts defined for Continuous TTCN-3 (C TTCN-3) , an extension of the standardized test specification language . Thus, we treat signals as streams and integrate the notion of signal properties with the notion of stream templates like they are already defined in C TTCN-3. Moreover, we provide a formal foundation for C TTCN-3 streams, for a selected set of signal properties and for their integration in C TTCN-3.
Lecture Notes in Computer Science, 2013
Model-based testing is a recognized method for testing the functionality of a system under test. ... more Model-based testing is a recognized method for testing the functionality of a system under test. However, it is not only the functionality of a system that has to be assessed. Also the security aspect has to be tested, especially for systems that provide interfaces to the Internet. In order to find vulnerabilities that could be exploited to break into or to crash a system, fuzzing is an established technique in industry. Model-based fuzzing complements model-based testing of functionality in order to find vulnerabilities by injecting invalid input data into the system. While it focuses on invalid input data, we present a complementary approach called behavioral fuzzing. Behavioral fuzzing does not inject invalid input data but sends an invalid sequence of messages to the system under test. We start with existing UML sequence diagrams -e.g. functional test cases -and modify them by applying fuzzing operators in order to generate invalid sequences of messages. We present the identified fuzzing operators and propose a classification for them. A description of a case study from the ITEA-2 research project DIAMONDS as well as preliminary results are presented.
Continuous TTCN-3
Proceedings of the 2006 international workshop on Software engineering for automotive systems, 2006
2010 17th IEEE International Conference and Workshops on Engineering of Computer Based Systems, 2010
Since software systems become more and more complex, the efforts for developing, documenting and ... more Since software systems become more and more complex, the efforts for developing, documenting and executing meaningful test cases increases. Testing is a vital, but time-and resource-consuming activity. To avoid running out of time or budget, new test methodologies had to be established in order to increase reliable, yet maintainable test scenarios. In the last years the Model-Driven idea matures to the most promising approaches to solve current problems in the software development domain. Model-Based Testing adopts these concepts to exploit their benefits for testing area. In this paper an integrated tool chain (called FOKUS!MBT) is discussed, to enable a Model-based development of testing scenarios. It is based on a canonical metamodel for testing concerns and a service-oriented model storage and exchange infrastructure, that allows a flexible, yet extensible adaptation to different test process requirements. Its premise is to establish a tooling architecture for the specification and development of a domain-independent Model-based testing scenario.
This paper describes an approach for risk-based testing of Bluetooth functionality in an automoti... more This paper describes an approach for risk-based testing of Bluetooth functionality in an automotive environment, recentlystudied as part of theITEA-2 research project DIAMONDS.
2009 14th IEEE International Conference on Engineering of Complex Computer Systems, 2009
The European Space Agency (ESA) as many other companies is interested in capitalizing its busines... more The European Space Agency (ESA) as many other companies is interested in capitalizing its business assets. With the space programmes often lasting 10 to 20 years, the software system migration problems arise frequently. The Object Management Group promotes the Model Driven Architecture (MDA) concept and proposes the Architecture Driven Modernization (ADM) approach for model-based platform migration. SOFTEAM, Fraunhofer FOKUS and GTI6 performed an ESA-funded study on Round Trip Engineering for Space Systems. During this study the state-of-the-art methods and tools for ADM and MDA were combined with state-of-the-art model based testing (MBT) approaches to safeguard the modernization process. Both techniques were assessed by applying them to a real-life use case -the migration and testing of a distributed archive and versioning system. In this article we overview the combined platform migration and testing methodology used in the project and summarize our experience during its application to the ESA's File Archive System. We specially focus on real life experience with MBT and discuss lessons learned.
Lecture Notes in Computer Science
Test processes in the automotive industry are tool-intensive and affected by technologically hete... more Test processes in the automotive industry are tool-intensive and affected by technologically heterogeneous test infrastructures. In the industrial practice a product has to pass tests at several levels of abstraction such as Model-in-the-Loop (MIL), Software-in-the-Loop (SIL) and Hardware-in-the-Loop (HIL) tests. Different test systems are applied for this purpose (e.g. dSPACE MTest, dSPACE Automation Desk, National Instruments Teststand) and almost each test system requests its own proprietary test description language. The exchange of tests between different test systems and the reuse of tests between different test levels is normally not possible. Efforts to integrate these heterogeneous test environments, to address test exchange in a general manner and to standardize and harmonize the existing language environment are still at the beginning and not tailored towards the requirements of the automotive domain. To keep the whole development and test process efficient and manageable, the definition of an integrated and seamless approach is required. TestML-the test exchange language we present in this articleis defined to overcome the technological obstacles (different test language syntax and semantics, different data formats and interface descriptions) that almost automatically accompany the application of heterogeneous test tools and test infrastructures. TestML supports the exchange of tests between different test notations in a heterogeneous tool environment. In this paper, we introduce the XML schema of TestML and demonstrate the efficiency of the interchange format by giving examples from the model-based development of electronic control units. Tool support is illustrated by an application with Simulink/Stateflow.
2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops, 2013
Fuzz testing or fuzzing is interface robustness testing by stressing the interface of a system un... more Fuzz testing or fuzzing is interface robustness testing by stressing the interface of a system under test (SUT) with invalid input data. It aims at finding security-relevant weaknesses in the implementation that may result in a crash of the systemunder-test or anomalous behavior. Fuzzing means sending invalid input data to the SUT, the input space is usually huge. This is also true for behavioral fuzzing where invalid message sequences are submitted to the SUT. Because systems getting more and more complex, testing a single invalid message sequence becomes more and more time consuming due to startup and initialization of the SUT. We present an approach to make the test execution for behavioral fuzz testing more efficient by generating test cases at runtime instead of before execution, focusing on interesting regions of a message sequence based on a previously conducted risk analysis and reducing the test space by integrating already retrieved test results in the test generation process.
Communications in Computer and Information Science, 2014
Most software vulnerabilities arise from common causes and the top 10 cause account for about 75%... more Most software vulnerabilities arise from common causes and the top 10 cause account for about 75% of all software vulnerabilities • More than 90% of the vulnerabilities are caused by known causes • The number of vulnerabilities being discovered in applications is far greater than the number of vulnerabilities discovered in operating systems • Due to SEI and to McAfee, majority of security breaches is due to software faults © DIAMONDS Consortium 2010-2013 • Security engineering is increasingly challenged by the openness, dynamics, and distribution of networked systems • Most verification and validation techniques for security have been developed in the framework of static or known configurations, with full or well-defined control of each component of the system • This is not sufficient in networked systems, where control and observation of remote (sub) systems are dynamically invoked over the network • DIAMONDS -Development and Industrial Application of Multi-Domain Security Testing Technologies -challenges the: Combination of active and passive security testing Usage of fuzz tests (for unknown issues) and functional tests (for security measures) Combination of risk analysis and test generation Integration of automated test generation, test execution and monitoring
International Journal on Software Tools for Technology Transfer, 2008
TTCN-3 has gained increasing significance in recent years. It was originally developed to fit the... more TTCN-3 has gained increasing significance in recent years. It was originally developed to fit the needs of testing software-based applications and systems in the telecommunication industry and has shown its applicability to a wide range of other industrial domains in the mean time. TTCN-3 provides platform-independent, universal and powerful concepts to describe tests, especially for discrete, interactive systems. However, TTCN-3 addresses systems with discrete input and output characteristics only. The lack of powerful means that reasonably allow specifying and evaluating continuous data flow makes TTCN-3 sufficient neither for the automotive industry nor for other industries that deal with highly complex software-based control systems. This paper introduces the notion of streams, stream ports and stream templates to TTCN-3. It revises the initial design of continuous TTCN-3, a TTCN-3 extension for testing continuous or hybrid systems [20,21] and demonstrates the applicability for a case study that is typical for testing embedded control systems in the automotive industry. I. Schieferdecker • J. Grossmann (B) Fraunhofer FOKUS,
Auch wenn ihr durch die Prägung des Begriffs "Model-Driven Architecture" (MDA) durch die OMG zusä... more Auch wenn ihr durch die Prägung des Begriffs "Model-Driven Architecture" (MDA) durch die OMG zusätzliche Aufmerksamkeit zugekommen ist, hat die modellbasierte Entwicklung in vielen Bereichen von der Geschäftsprozessmodellierung bis hin zur Beschreibung von eingebetteten Steuerungssystemen Anwendung gefunden und geht dabei in den eingesetzten Techniken und Verfahren über die Trennung von plattformunabhängigen und plattformspezifischen Systembeschreibungen und den Übergang dazwischen hinaus. Zentrales Merkmal der modellbasierten Entwicklung ist dabei der Einsatz von Modellen, die sich an der Problem-anstatt der Lösungsdomäne orientieren. Dies bedingt einerseits die Bereitstellung anwendungsorientierter Modelle (z.B. Matlab/Simulink-artige für regelungstechnische Problemstellungen, Statechart-artige für reaktive Anteile) und ihrer zugehörigen konzeptuellen (z.B. Komponenten, Signal, Nachrichten, Zustände) und semantischen Aspekte (z.B. synchroner Datenfluss, ereignisgesteuerte Kommunikation). Andererseits bedeutet dies auch die Abstimmung auf die jeweilige Entwicklungsphase, mit Modellen von der Anwendungsanalyse (z.B. Beispielszenarien, Schnittstellenmodelle) bis hin zur Implementierung (z.B. Bus-oder Task-Schedules, Implementierungstypen). Für eine durchgängige modellbasierte Entwicklung ist daher im Allgemeinen die Verwendung eines Modell nicht ausreichend, sondern der Einsatz einer Reihe von abgestimmten Modellen für Sichten und Abstraktionen des zu entwickelnden Systems (z.B. funktionale Architektur, logische Architektur, technische Architektur, Hardware-Architektur) nötig. Durch den Einsatz problem-statt lösungszentrierter Modelle kann in jedem Entwicklungsabschnitt von unnötigen Festlegungen abstrahiert werden, während besonders wichtige und kritische Aspekte explizit und frühzeitig modelliert werden (z.B. Zeit, Prioritäten oder Kommunikationsaspekte). Die dadurch ermöglichte Anwendung analytischer und generativer Verfahren auf diesen Modellen ermöglicht die effiziente Entwicklung hochqualitativer Software. Modellbasierte Vorgehensweisen haben gerade in der Softwareentwicklung in den letzten Jahren deutlich an Bedeutung gewonnen. Gerade im Bereich eingebetteter Software (z.B. Automotive oder Avionic Software Engineering) erfährt der Einsatz von domänenspezifischen Modellierungswerkzeugen in der Softwarenetwicklung zunehmend an Verbreitung. Wesentlich dazu haben dabei die Weiterentwicklung von Sprachen für aufgabenspezifische Modelle (z.B. synchroner Datenfluss) und dazugehörige Werkzeugen für spezialisierte Bereiche (z.B. Regelungs-und Steuerungsalgorithmen, Anlagensteuerung) und die Verbesserung der Ent-wicklungswerkzeuge, vor allem hinsichtlich Implementierungsqualität, Bedienkomfort und Analysemächtigkeit beigetragen. Trotzdem sind im Kontext der modellbasierten Entwicklung noch viele, auch grundlegende Fragen offen, insbesondere im Zusammenhang mit der Durchgängigkeit. Die in diesen Tagungsband zusammengefassten Papiere stellen zum Teil gesicherte Ergebnisse, Work-In-Progress, industrielle Erfahrungen und innovative Ideen aus diesem Bereich zusammen und erreichen damit eine interessante Mischung theoretischer Grundlagen und praxisbezogener Anwendung. V Genau wie beim ersten, im Januar 2005 erfolgreich durchgeführten Workshop sind damit wesentliche Ziele dieses Workshops erreicht: -Austausch über Probleme und existierende Ansätze zwischen den unterschiedlichen Disziplinen (insbesondere Elektro-und Informationstechnik, Maschinenwesen/Mechatronik und Informatik) -Austausch über relevante Probleme in der Anwendung/Industrie und existierende Ansätze in der Forschung -Verbindung zu nationalen und internationalen Aktivitäten (z.B. Initiative des IEEE zum Thema Model-Based Systems Engineering, GI-AK Modellbasierte Entwicklung eingebetteter Systeme, GI-FG Echtzeitprogrammierung, MDA Initiative der OMG) Die Themengebiete, für die dieser Workshop gedacht ist und fachlich sehr gut abgedeckt sind, sich dieses Jahr (mit Ausnahmen) aber sehr stark auf den automotiven Bereich konzentrieren, fokussieren auf Teilaspekte modellbasierter Entwicklung eingebetteter Softwaresysteme. Darin enthalten sind unter anderem: -Domänenspezifische Ansätze zur Modellierung von Systemen (z.B. Avionik, Railway, Automotive, Produktions-und Automatisierungstechnik) -Durchgängigkeit und Integration von Modellen für eingebettete Systeme -Modellierung spezifischer Eigenschaften eingebetteter Systeme (z.B. Echtzeiteigenschaften, Robustheit/Zuverlässigkeit, Ressourcenmodellierung) -Konstruktiver Einsatz von Modellen (Generierung und Evolution) -Modellbasierte Validierung und Verifikation Das Organisationskomitee ist der Meinung, dass mit den Teilnehmern aus Industrie, Werkzeugherstellern und der Wissenschaft die bereits 2005 erfolgte Community-Bildung erfolgreich weitergeführt wurde, und damit demonstriert, dass eine solide Basis zur Weiterentwicklung des noch jungen Felds modellbasierter Entwicklung eingebetteter Systeme existiert. Die Durchführung eines erfolgreichen Workshops ist ohne vielfache Unterstützung nicht möglich. Wir danken daher den Mitarbeitern von Schloss Dagstuhl und natürlich unseren Sponsoren.
Lecture Notes in Computer Science, 2007
TTCN-3 has gained increasing significance in recent years. Originally developed to fit the needs ... more TTCN-3 has gained increasing significance in recent years. Originally developed to fit the needs for testing software-based applications and systems in the telecommunication industry, TTCN-3 has shown its applicability to a wide range of other industrial domains in the mean time. TTCN-3 provides platform-independent, universal and powerful concepts to describe tests -especially for discrete, interactive systems -on different levels of abstraction. However, TTCN-3 addresses systems with discrete input and output characteristics only. In the automotive industry -as well as in other industries that deal with highly complex software-based control systems -this is not sufficient. Control systems often interact with their environment trough sensors and actuators using continuous signals. A test environment that adequately supports the specification, execution and evaluation of tests for embedded control systems has to provide concepts to handle this kind of signals. Moreover it has to support the test engineer with suitable abstractions that ease signal specification and signal evaluation.
Uploads
Papers by Jürgen Großmann