Assurance optimization

IEEE Open Journal of Systems Engineering, 2022

System verification activities (VAs) are used to identify potential errors and corrective activities (CAs) are used to eliminate those errors. However, existing math-based methods to plan verification strategies do not consider decisions to implement VAs and perform CA jointly, ignoring their close interrelationship. In this article, we present a joint verification–correction model to find optimal joint verification–correction strategies (JVCSs). The model is constructed so that both VAs and CAs can be chosen as dedicated decisions with their own activity spaces. We adopt the belief model of Bayesian networks to represent the impact of VAs and CAs on verification planning and use three value factors to measure the performance of JVCSs. Moreover, we propose an order-based backward induction approach to solve for the optimal JVCS by updating all verification state values. A case study was conducted to show that our model can be applied to effectively solve the verification planning problem.

2007 Proceedings - Annual Reliability and Maintainability Sympsoium, 2007

Safety, Security and Reliability of complex systems are the three interacting and most important risk related factors. In many cases of failure event, the Security function assumes charge, and manages the failure event and its resolution. But does the Security function consistently apply the optimal failure resolution methods? We propose that several organizational functions, including Information Security (IS), should analyze, manage, and resolve each case of failure in a coordinated effort, based on the failure classification and prioritization, and then apply appropriate Corrective Actions (CA).

Reliability Engineering & System Safety, 1992

The development of a method, INTENT, for estimating probabilities associated with decisionbased errors is presented. These errors are not ordinarily incorporated into probabilistic risk assessments (PRAs) due to both the difficulty in postulating such errors and to the lack of a method for estimating their probabilities from existing data. By failing to include deeisionbased errors in their analyses, most PRA practitioners seriously underestimate the true contribution of human actions to systems failure. This paper attempts to extend the identification of such errors and to quantify them. Two sources, Nuclear Computerized Library for Assessing Reactor Reliability (NUCLARR) and licensee event reports (LERs) were reviewed and two methods, HSYS and SNEAK, were used to identify a generic list of twenty potential errors which may be manifest as erroneous acts. Four categories of influence emerged from the data: consequence, attitudes, response set, and dependency. Corresponding human error probabilities (HEPs) for each error were generated by expert judgment methods. Lower and upper bounds for the HEPs for each error were determined by positing a situation reflecting optimized and degraded performance shaping factors, respectively. To allow analysts the opportunity to refine these extreme HEP values when evaluating a particular scenario of interest, normalization procedures were conducted and generic importance weights were computed for each of 11 performance shaping factors (PSFs) believed to affect the 20 deeisionbased errors. It is believed by the authors that PSFs constitute a performance influence which, in some cases, such as in that for training, can serve to either augment or reduce the intellectual resources used by people to successfully accomplish tasks. These derived importance weights are used in conjunction with situation specific PSF ratings to compute a composite PSF score which, in turn, is mapped onto an HEP distribution. Distribution assumptions are presented and a function defining the relationship between composite PSF scores and HEPs is presented for use by the analyst.

11th ACM SIGSOFT …, 2003

We present an integrated approach to risk assessment and risk mitigation that is well suited to planning the development of complex software systems. Our integrated approach is able to derive estimates of the costs and benefits (in terms of qualities of the developed product) at the time of planning a development. It accommodates both process knowledge (the efficacy of development practices) and product knowledge (the logical structure of the system under development). Functional and non-functional aspects of sofiare can also be accommodated, and trades made among them. Optimization -selecting the best suite of process steps and design choices to maximize the expectation of success while remaining within budgetbecomes possible. The key to this is the integration of two complementary methods for reasoning about risks. One set of methods is that found in the area of Probabilistic Risk Assessment, specifically its methods ,for reasoning over logical fault trees. The other set of methods come from an earlylifecycle risk assessment and risk mitigation planning method that we have been developing and applying to spacecraft technology. The integration of the two methods we call "Probabilistic Risk Reduction", to draw attention to its probabilistic treatment of risk and explicit consideration of what can be done to reduce it.

2009

In a rapid development environment, the very identification of viable solutions forms part of the project, in addition to its implementation. The pressures to "find failure fast" require that candidate solutions must be evaluated on the run, requiring at least a preliminary and partial implementation whose chief purpose is to identify, assess and suitably control associated risks. This renders the traditional linear planning and control methodologies unsuitable for such work. In this paper, we developed a process which systematically analyses and prioritizes a series of previously selected candidate solutions using an AHP-based approach in which time, cost and quality criteria typically dominate. A quantitative risk assessment is performed on the highest rated candidate, leading either to its rejection, (and the initiation of a similar investigation into the next most preferred option), or the identification of a refinement believed to reduce the risk to acceptable levels. Since these refinements usually affect either cost, time or both, a re-calculation of the AHP is necessary, leading to an iterative "prioritize-plan-implement-test-reprioritize" process which provides a non-linear identification and implementation of the most promising solution from a cost/schedule/quality and risk perspective. The method highlights the use of Work Breakdown Structure fragments, the appropriate aggregation of risk, dynamic decision-making and flexible change management protocols. It also relies critically on the effective communication and cooperation between planners and decision-makers operating at the commercial / technical interface.

Advances in Decision Sciences, 2012

Comparatively few of the vast amounts of decision analytical methods suggested have been widely spread in actual practice. Some approaches have nevertheless been more successful in this respect than others. Quantitative decision making has moved from the study of decision theory founded on a single criterion towards decision support for more realistic decision-making situations with multiple, often conflicting, criteria. Furthermore, the identified gap between normative and descriptive theories seems to suggest a shift to more prescriptive approaches. However, when decision analysis applications are used to aid prescriptive decision-making processes, additional demands are put on these applications to adapt to the users and the context. In particular, the issue of weight elicitation is crucial. There are several techniques for deriving criteria weights from preference statements. This is a cognitively demanding task, subject to different biases, and the elicited values can be heavily dependent on the method of assessment. There have been a number of methods suggested for assessing criteria weights, but these methods have properties which impact their applicability in practice. This paper provides a survey of state-of-the-art weight elicitation methods in a prescriptive setting.

Computing in Civil Engineering (2012), 2012

Decision-making in fields such as politics, engineering and healthcare, shapes the world and how it evolves. Both public and private organizations face challenges when making decisions. Two examples occurred with the Minnesota Department of Transportation in 2007 and the U.S. Department of Energy in 2008. Losing bidders for a bridge-rebuilding contract and a liquid-waste cleanup project, respectively, protested the agencies' awards on the basis of evaluation methods and selection criteria. Multi-evaluator multi-criterion (MEMC) decision making can be controversial if bias or uncertainty find their way into the final decision.

IEEE Transactions on Systems, Man, and Cybernetics, 1990

Modeling-to-generate alternatives (MGA) is a technique for using mathematical programming models to generate a small number of different solutions for the decision maker to consider when dealing with complex, incompletely defined problems. The logic of MGA is presented in the context of concerns about the limitations of mathematical models and limitations of the human decisionmakers who use them. Arguments and experimental evidence are presented to support the assumption that the human-machine decisionmaking system will perform better when the human is presented with a few, different alternatives than when presented a homogeneous set of alternatives as might result from sensitivity analysis.

Systems of Systems - Engineering, Modeling, Simulation and Analysis [Working Title], 2020

The chapter presents an extension of a previous method for decision support under risk. The decision-making process is modeled by a multi-criteria optimization problem, in which the individual evaluation functions represent the results of decisions in several possible scenarios with associated risks. The decision support method is an interactive decision-making process. The choice is made by solving the problem depending on the control parameters that define the aspirations of the decision maker as well as on an evaluation of the obtained solutions. The decision maker selects a set of parameters representing various risks’ impacts that influences a solution, and then he/she evaluates the obtained solution by accepting or rejecting it. In another case, the decision maker selects a new value and the problem is solved again for the new parameter. In this chapter, an example of supporting decision-making under risk is presented.