Integrated safety and security assurance for complex systems is difficult for many technical and ... more Integrated safety and security assurance for complex systems is difficult for many technical and socio-technical reasons, such as mismatched processes, inadequate information, differing use of language and philosophies, etc. Many co-assurance techniques rely on disregarding some of these challenges to present a unified methodology. Even with this simplification, no methodology has been widely adopted, primarily because this approach is unrealistic when met with the complexity of real-world system development. This paper presents an alternate approach by providing a Safety-Security Assurance Framework (SSAF) based on a core set of assurance principles. This is done so that safety and security can be co-assured independently, as opposed to a unified co-assurance, which has been shown to have significant drawbacks. This also allows for separate processes and expertise from practitioners in each domain. In this structure, the focus is shifted from simplified unification to integration t...
Assurance cases are often required to certify critical systems. The use of formal methods in assu... more Assurance cases are often required to certify critical systems. The use of formal methods in assurance can improve automation, increase confidence, and overcome errant reasoning. However, assurance cases can never be fully formalised, as the use of formal methods is contingent on models that are validated by informal processes. Consequently, assurance techniques should support both formal and informal artifacts, with explicated inferential links between them. In this paper, we contribute a formal machine-checked interactive language, called Isabelle/SACM, supporting the computer-assisted construction of assurance cases compliant with the OMG Structured Assurance Case Meta-Model. The use of Isabelle/SACM guarantees well-formedness, consistency, and traceability of assurance cases, and allows a tight integration of formal and informal evidence of various provenance. In particular, Isabelle brings a diverse range of automated verification techniques that can provide evidence. To valida...
Assurance cases (ACs) are often required to certify critical systems. The use of integrated forma... more Assurance cases (ACs) are often required to certify critical systems. The use of integrated formal methods (FMs) in assurance can improve automation, increase confidence, and overcome errant reasoning. However, ACs can rarely be fully formalised, as the use of FMs is contingent on models that are validated by informal processes. Consequently, assurance techniques should support both formal and informal artifacts, with explicated inferential links between them. In this paper, we contribute a formal machine-checked interactive language for the computer-assisted construction of ACs called Isabelle/SACM. The framework guarantees well-formedness, consistency, and traceability of ACs, and allows a tight integration of formal and informal evidence of various provenance. To validate Isabelle/SACM, we present a novel formalisation of the Tokeneer benchmark, verify its security requirements, and form a mechanised AC that combines the resulting formal and informal artifacts.
Safety-critical systems developed upon SPLE approach have to address safety standards, which esta... more Safety-critical systems developed upon SPLE approach have to address safety standards, which establish guidance for analyzing and demonstrating dependability properties of the system at different levels of abstraction. However, the adoption of an SPLE approach for developing safety-critical systems demands the integration of safety engineering into SPLE processes. Thus, variability management in both system design and dependability analysis should be considered through SPLE life-cycle. Variation in design and context may impact on dependability properties during Hazard Analysis and Risk Assessment (HARA), allocation of functional and non-functional safety requirements, and component fault analysis. This paper presents DEPendable-SPLE, a modelbased approach that extends traditional SPLE methods, to support variability modeling/management in dependability analysis. The approach is illustrated in a case study from the aerospace domain. As a result, the approach enabled efficient management of the impact of design and context variations on HARA and component fault modeling.
When creating an assurance justification for a critical system, the focus is often on demonstrati... more When creating an assurance justification for a critical system, the focus is often on demonstrating technical properties of that system. Complete, compelling justifications also require consideration of the processes used to develop the system. Creating such justifications can be an onerous task for systems using complex processes and highly integrated tool chains. In this paper we describe how process models can be used to automatically generate the process justifications required in assurance cases for critical systems. We use an example case study to illustrate an implementation of the approach. We describe the advantages that this approach brings for system assurance and the development of critical systems. Con: components trusted software components: {trusted software components} Con: enviroProps assumed environmental properties: {assumed environmental properties} Goal: verifResults Results of formal verification demonstrate {formal property} is satisfied Goal: formalConf There is sufficient confidence in the formal verification results Sol: verifResults {formal verification results for {formal property}} Con: platformProps properties of system platform: {assumed platform properties} Goal: propSat {formal property} is satisfied in the system model Goal: verification Verification using {technique} gives trustworthy results {technique} process
assurance claims can be modelled in SACM. Such claims abstract from the details of the argumentat... more assurance claims can be modelled in SACM. Such claims abstract from the details of the argumentation (e.g. what is the subject for which the claim is made), its constituent properties and elements etc. Instantiating the claim means replacing the abstract references within the claim with concrete ones, requisitioned from the appropriate ODE elements within the rest of the DDI. Thus, in the case of Figure 16, the ‘Instantiation Script’ elements attached to each node of the DAG control how the abstract references in the descriptions, denoted using the ‘{‘ ‘}’ brackets, are replaced. Specifically, in the top node, {X} is replaced with ‘System A’, being the name of the ODE element being referenced. In the supporting node, properties of constituent elements of X form an assertion supporting the previous claim. The constituent elements can be referenced using a universal quantifier that ranges over the domain of constituent elements of X. In practical terms, this can be realized using a pr...
Abstract: In this paper, we argue that informal logic argument schemes have important roles to pl... more Abstract: In this paper, we argue that informal logic argument schemes have important roles to play in safety argument construction and reviewing process. Ten commonly used reasoning schemes in computer system safety domain are proposed. The role of informal logic dialogue games in system safety arguments reviewing is also discussed and our intended work in this area is proposed. It is anticipated that this work will contribute toward the development of safety arguments and help to move forward the interplay between research in informal logic and research in computer system safety engineering. Resumé: Dans cet article nous soutenons que les schèmes de la logique non formelle jouent des rôles importants dans la construction d'arguments employés dans des systèmes de sécurité en informatique. Nous décrivons dix de ces schèmes de raisonnement qui sont couramment utilisés. Nous discutons du rôle des jeux de dialogue en logique non formelle joué dans ces systèmes, et de nos travaux p...
The increasing role of Systems of Systems (SoS) in safetycritical applications establishes the ne... more The increasing role of Systems of Systems (SoS) in safetycritical applications establishes the need for methods to ensure their safe behaviour. One approach to ensuring this is by means of safety policy-a set of rules that all the system entities must abide by. This paper proposes simulation as a means to evaluate the effectiveness of such a policy. The requirements for simulation models are identified, and a means for decomposing high-level policy goals into machine-interpretable policy rules is described. It is then shown how the enforcement of policy could be integrated into a simple agent architecture based around a blackboard. Finally, an approach to evaluating the safety of a system based on simulation runs is outlined.
2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), 2015
Safety cases present the arguments and evidence that can be used to justify the acceptable safety... more Safety cases present the arguments and evidence that can be used to justify the acceptable safety of a system. Many secondary factors such as the tools used, the techniques applied, and the experience of the people who created the evidence, can affect an assessor's confidence in the evidence cited by a safety case. One means of reasoning about this confidence and its inherent uncertainties is to present a 'confidence argument' that explicitly justifies the provenance of the evidence used. In this paper, we propose a novel approach to automatically construct these confidence arguments by enabling assessors to provide individual judgements concerning the trustworthiness and the appropriateness of the evidence. The approach is based on Evidential Reasoning and enables the derivation of a quantified aggregate of the overall confidence. The proposed approach is supported by a prototype tool (EviCA) and has been evaluated using the Technology Acceptance Model.
Context: Many critical systems must comply with safety standards as a way of providing assurance ... more Context: Many critical systems must comply with safety standards as a way of providing assurance that they do not pose undue risks to people, property, or the environment. Safety compliance is a very demanding activity, as the standards can consist of hundreds of pages and practitioners typically have to show the fulfilment of thousands of safety-related criteria. Furthermore, the text of the standards can be ambiguous, inconsistent, and hard to understand, making it difficult to determine how to effectively structure and manage safety compliance information. These issues become even more challenging when a system is intended to be reused in another application domain with different applicable standards. Objective: This paper aims to resolve these issues by providing a metamodel for the specification of safety compliance needs for critical systems. Method: The metamodel is holistic and generic, and abstracts common concepts for demonstrating safety compliance from different standards and application domains. Its application results in the specification of Òreference assurance frameworksÓ for safety-critical systems, which correspond to a model of the safety criteria of a given standard. For validating the metamodel with safety standards, parts of several standards have been modelled by both academic and industry personnel, and other standards have been analysed. We further augment this with feedback from practitioners, including feedback during a workshop. Results: The results from the validation show that the metamodel can be used to specify safety compliance needs for aerospace, automotive, avionics, defence, healthcare, machinery, maritime, oil and gas, process industry, railway, and robotics. Practitioners consider that the metamodel can meet their needs and find benefits in its use. Conclusion: The metamodel supports the specification of safety compliance needs for most critical computer-based and software-intensive systems. The resulting models can provide an effective means of structuring and managing safety compliance information.
2014 Brazilian Symposium on Computing Systems Engineering, 2014
Software product lines (SPL) have been successfully used in the development of automotive and avi... more Software product lines (SPL) have been successfully used in the development of automotive and avionics critical embedded systems. Hazards and their causes may change according to the selection of variants in a particular SPL product. Thereby, lower-level assets like fault trees and FMEA (Failure Modes and Effects Analysis) cannot be reused because they are dependent upon the selection of product variants. In this paper, model-based safety analysis techniques and SPL variability management tools are used together to reduce the effort of product safety analysis by: reusing SPL hazard analysis, and providing automatic safety analysis for each SPL product. Therefore, we propose a model-based approach to support the generation of safety analysis assets for multiple safety-critical SPL products. The proposed approach is illustrated using the Hephaestus variability management tool and the HiP-HOPS model-based safety analysis tool to generate fault trees and FMEA for the products of an automotive hybrid braking system SPL. Applying the approach reduced the effort to perform product safety analysis.
System assurance cases are used to demonstrate confidence in system properties of interest (e.g. ... more System assurance cases are used to demonstrate confidence in system properties of interest (e.g. safety and/or security). They are key artefacts for safety and/or security acceptance for systems before they become operational. Cyber-Physical Systems (CPS) form a new technological frontier for their vast economic and societal potentials in various domains. CPS are often safety-critical systems. Thus, their safety and/or security need to be assured using system assurance cases. However, due to the open and adaptive nature of CPS, the need for system assurance at runtime is imperative. Therefore, assurance cases are expected to be exchanged, integrated and verified at runtime to ensure the dependability of CPS when they intend to execute a cooperative behaviour. In this position paper, we identify the importance of model-based system assurance, we discuss the paradigm shift of assurance cases from being manually created artefacts to (semi-)automatically created models. We discuss the a...
There are many performance based techniques that aim to improve the safety of neural networks for... more There are many performance based techniques that aim to improve the safety of neural networks for safety critical applications. However, many of these approaches provide inadequate forms of safety assurance required for certification. As a result, neural networks are typically restricted to advisory roles in safetyrelated applications. Neural networks have the ability to operate in unpredictable and changing environments. It is therefore desirable to certify them for highlydependable roles in safety critical systems. This paper outlines the safety criteria which are safety requirements for the behaviour of neural networks. If enforced, the criteria can contribute to justifying the safety of ANN functional properties. Characteristics of potential neural network models are also outlined and are based upon representing knowledge in interpretable and understandable forms. The paper also presents a safety lifecycle for artificial neural networks. This lifecycle focuses on managing behaviour represented by neural networks and contributes to providing acceptable forms of safety assurance.
In the operation of safety-critical systems, the sequences by which failures can lead to accident... more In the operation of safety-critical systems, the sequences by which failures can lead to accidents can be many and complex. This is particularly true for the emerging class of systems known as systems of systems, as they are composed of many distributed, heterogenous and autonomous components. Performing hazard analysis on such systems is challenging, in part because it is difficult to know in advance which of the many observable or measurable features of the system are important for maintaining system ...
Abstract: Safety case development is not a post-development activity, rather it should occur thro... more Abstract: Safety case development is not a post-development activity, rather it should occur throughout the system development lifecycle. The key components in a safety case are safety arguments. Too often, safety arguments are constructed without proper reasoning. Inappropriate reasoning in safety arguments could undermine a system’s safety claims, which in turn contributes to safety-related failures of the system. To address this, we argue that informal logic argument schemes have important roles to play in safety arguments construction and review process. Ten commonly used reasoning schemes in computer system safety domain are proposed against the safety engineering literature. The role of informal logic dialogue games in computer system safety arguments reviewing is also discussed and a dialectical model for safety argument review is proposed. It is anticipated that this work will contribute toward the development of computer system safety arguments, and help to move forward the...
In Europe, over recent years, the responsibility for ensuring system safety has shifted onto the ... more In Europe, over recent years, the responsibility for ensuring system safety has shifted onto the developers and operators to construct and present well reasoned arguments that their systems achieve acceptable levels of safety. These arguments (together with supporting evidence) are typically referred to as a “safety case”. This paper describes the role and purpose of a safety case. Safety arguments within safety cases are often poorly communicated. This paper presents a technique called GSN (Goal Structuring Notation) that is increasingly being used in safety-critical industries to improve the structure, rigor, and clarity of safety arguments. The paper also describes a number of extensions, based upon GSN, which can be used to assist the maintenance, construction, reuse and assessment of safety cases. The aim of this paper is to describe the current industrial use and research into GSN such that its applicability to other types of Assurance Case, in addition to safety cases, can al...
Quantitative modelling and analysis is common in safety engineering, but it is often criticised. ... more Quantitative modelling and analysis is common in safety engineering, but it is often criticised. Objections include the difficulty in acquiring probabilities (e.g. for human error), the dubious assumptions often needed to manipulate them (e.g. independence of events), and the inherent uncertainty involved in making decisions based on probabilistic predictions. Clearly, poor predictions are of little value and may be dangerous. Faced with this danger, many people respond by eliminating quantities altogether. This is a trap, as we have no guarantee that the resulting model or predictions will be better; indeed, the subtlety of expression offered by numerical probabilities has been lost. This paper discusses some alternatives to the non-quantitative trap, and explores their significance for the issue of safety case assurance.
The Unmanned Systems Safety Guide for DoD Acquisition (henceforth, “the guide”) is intended to he... more The Unmanned Systems Safety Guide for DoD Acquisition (henceforth, “the guide”) is intended to help safety engineers achieve safety in unmanned systems acquisition projects. Given the rising number of such projects, this is a worthy goal. The guide contains some sound advice, but it also has a number of weaknesses. In particular the core elements of the guide (the Top Level Mishaps and the Precepts) are poorly defined, and the choice of what to include is odd. Using the guide in support of a risk-based safety process is difficult because of the lack of explicit rationale. The way that the guide is structured may make it cumbersome to maintain in future. We identify a number of ways in which the guide could be improved, and highlight some key opportunities for future development. Introduction The Unmanned Systems Safety Guide for DoD Acquisition (ref. 1) was issued in July 2007. It aims, in its own words, “to ensure the design and development of UMSs [Unmanned Systems] that incorpora...
In software engineering the role of software architecture as a means of managing complexity and a... more In software engineering the role of software architecture as a means of managing complexity and achieving emergent qualities such as modifiability is increasingly well understood. In this paper we demonstrate how many principles from the field of software architecture can be brought across to the field of safety case management in order to help manage complex safety cases. Traditional approaches to certification of modular systems as a statically defined configuration of components can result in a large certification overhead being associated with any module update or addition. A more promising approach is to attempt to establish a modular, compositional, approach to constructing safety cases that has a correspondence with the modular structure of the underlying architecture. This paper establishes the mechanisms for managing and representing safety cases as a composition of safety case 'modules'. Having defined the concept of a modular safety case, the paper also describes ...
Aerospace Recommended Practices (ARPs) 4754 and 4761 introduce the concept of preliminary system ... more Aerospace Recommended Practices (ARPs) 4754 and 4761 introduce the concept of preliminary system safety assessment (PSSA) as a key stage in the safety process for systems on civil aircraft. PSSA is intended to follow functional hazard assessment (FHA). Its purpose is to assist in validating a proposed system architecture and to allocate (derived) safety requirements to components of that architecture. Although the ARPs claim to represent “best practise” some of their recommendations, including the conduct of PSSA, are novel, and it is not always clear how to interpret and apply them. The purpose of this paper is to give some guidelines on the conduct of PSSA, based on our experience of assisting a number of organisations in developing safety processes in response to the ARPs.
Uploads
Papers by Tim Kelly