Papers by Nick Shallcross
Proceedings of the International Annual Conference of the American Society for Engineering Management, 2023
Every year, U.S. Army commands and senior leaders must defend their force structure requirements ... more Every year, U.S. Army commands and senior leaders must defend their force structure requirements during the Program Objective Memorandum (POM) process. Informing these resourcing decisions and planning efforts requires rigorous and repeatable analysis with a tailorable approach. Sufficiency analysis enables staff planners to identify right-sized force structure while estimating the risk to mission when force structure is insufficient. Fundamentally, sufficiency analysis answers: Is there a sufficient force structure (supply) to meet a set of projected mission demands using tailored readiness business rules? Three main inputs are required for sufficiency analysis, which include a plausible mission demand signal, force structure broken-down into unit of employment, and readiness business rules. This approach has been applied across Army calibrated force posture, Security Force Assistance Command, special operations commands, and other Joints elements. This presentation describes an objective, scalable, repeatable, and flexible sufficiency analysis, and optimization approach for future studies of continually evolving force structure priorities.
A value of information methodology for multiobjective decisions in quantitative set‐based design
Systems Engineering, 2021
Engineering complex systems is an exercise in sequential multiobjective decision making under unc... more Engineering complex systems is an exercise in sequential multiobjective decision making under uncertainty. One method for handling this complexity and uncertainty is set‐based design (SBD). SBD is a concurrent engineering and management methodology that develops, analyzes, and matures numerous design options, reducing risk and delivering higher value to the stakeholders and end users. SBD accomplishes this through controlled design space convergence which reduces uncertainty and prevents premature design decisions. While SBD has been the subject of numerous scholarly articles, there is limited research providing quantitative methodologies that inform decisions enabling design maturation and convergence. We present a value of information (VOI) based methodology for multiobjective decision problems, and demonstrate its applicability for SBD decisions. We apply Bayesian decision models and information value to inform multiobjective modeling and design maturation decisions. Research contributions include: 1) a framework integrating VOI into the SBD process, 2) a multiobjective VOI method assessing a higher‐resolution model's ability to reduce uncertainty, and 3) a means of informing modeling decisions by comparing multiple high resolutions models, given their usage cost and their potential to deliver information value. Finally, we demonstrate the inherent issues associated with premature decisions and traditional point‐based design approaches which run the risk of selecting an alternative that later proves infeasible.
Applied Sciences, 2021
System designers, analysts, and engineers use various techniques to develop complex systems. A tr... more System designers, analysts, and engineers use various techniques to develop complex systems. A traditional design approach, point-based design (PBD), uses system decomposition and modeling, simulation, optimization, and analysis to find and compare discrete design alternatives. Set-based design (SBD) is a concurrent engineering technique that compares a large number of design alternatives grouped into sets. The existing SBD literature discusses the qualitative team-based characteristics of SBD, but lacks insights into how to quantitatively perform SBD in a team environment. This paper proposes a qualitative SBD conceptual framework for system design, proposes a team-based, quantitative SBD approach for early system design and analysis, and uses an unmanned aerial vehicle case study with an integrated model-based engineering framework to demonstrate the potential benefits of SBD. We found that quantitative SBD tradespace exploration can identify potential designs, assess design feasi...
A Conditional Logistic Regression Predictive Model of World Conflict using Open Source Data
: Using open source data, this research formulates and constructs a suite of statistical models t... more : Using open source data, this research formulates and constructs a suite of statistical models that predict future transitions into and out of violent conflict and forecasts the regional and global incidences of violent conflict over a ten-year time horizon. A total of thirty predictor variables are tested and evaluated for inclusion in twelve conditional logistic regression models, which calculate the probability that a nation will transition from its current conflict state, either In Conflict or Not in Conflict, to a new state in the following year. These probabilities are then used to construct a series of nation-specific Markov chain models that forecast violent conflict, as well as yield insights into regional conflict trends out to year 2024 and beyond. The logistic regression models proposed in this study achieve training dataset accuracies of 88.76%, and validation dataset accuracies of 84.67%. Additionally, the Markov models achieve three year forecast accuracies of 85.16%...
Informing Program Management Decisions Using Quantitative Set-Based Design
IEEE Transactions on Engineering Management, 2021
System design is an exercise in sequential decisionmaking, with the objective of developing resil... more System design is an exercise in sequential decisionmaking, with the objective of developing resilient and affordable systems. Throughout the design process, engineering and program managers must balance several competing objectives, such as ensuring design feasibility, minimizing cost, schedule, and performance risk, while simultaneously achieving stakeholder value. Thus, engineering and program managers require design and analysis methods enabling complex and multiobjective design decisions under uncertainty. Unfortunately, there exists limited research providing quantitative methodologies specifically enabling program management decisions using quantitative set-based design. We, therefore, present a quantitative set maturation and uncertainty resolution decision methodology using value-focused multiobjective decision models and model-based engineering practices. This methodology assesses and quantifies uncertainty regarding stakeholder value, cost, requirements, and design maturity for each design set. These metrics facilitate the calculation of the program manager value, which when combined with design set feasibility entropy, enable tradeoff analysis informing design maturation and uncertainty resolution prioritization decisions. We develop and demonstrate our methodology using a model-based unmanned aerial vehicle case study implemented in the ModelCenter modeling environment. This methodology provides program managers an efficient, cost effective, and defensible approach to inform system design maturation and uncertainty resolution decisions enabling the development of resilient and affordable systems.
Set‐based design: The state‐of‐practice and research opportunities
Systems Engineering, 2020
Increasing system complexity has provided the impetus to develop new and novel systems engineerin... more Increasing system complexity has provided the impetus to develop new and novel systems engineering methodologies. One of these methodologies is set-based design (SBD), a concurrent design methodology well suited for complex systems subject to significant uncertainty. Since the 1990s, numerous private, public, and defense sector design programs have successfully implemented SBD. However, concerns regarding SBD's complexity, tendency toward qualitative methods, and lack of quantitative tools have limited its use. To address these issues, our research surveys 122 refereed journal articles and conference papers to assess SBD's state-of-practice and identify relevant research opportunities. To accomplish these tasks, we perform a structured literature review to identify and assess relevant and influential research. We found that SBD's state-of-practice relies heavily upon decision and tradespace analysis with increasing emphasis on uncertainty modeling and MBSE. We found that the majority of SBD research consists of quantitative methodologies focusing on component and small system applications. We also found that complex system applications used mostly qualitative methodologies. We identify SBD research opportunities for requirements development, MBSE, uncertainty modeling, multiresolution modeling, adversarial analysis, and program management. Finally, we recommend the development of a comprehensive SBD methodology and toolkit, suited for complex system design across all stages of the product development life cycle.
Predictive models of world conflict: accounting for regional and conflict-state differences
The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology, 2019
Developing nation-conflict forecasting models is of vital importance to international resource al... more Developing nation-conflict forecasting models is of vital importance to international resource allocation strategies that affect regional and worldwide security and stability. A nation in conflict may evolve very differently to one that is not. Using a unique set of open-source data, a suite of region-specific models are developed that predict nation-state transitions into and out of violent conflict. Conflict transition predictor variables differ not only between the regions, but also within each region dependent on if a nation is identified as in conflict or not. Accurate modeling of complex regional environments is achieved with parsimonious and interpretable models. While we ensure the causal variables of the previous study are considered for inclusion in the models developed, this paper focuses on the ability to accurately predict conflict over individual independent variable analysis.
Using value of information in quantitative set‐based design
Systems Engineering, 2021
Increasing system complexity requires that engineers, systems analysts, and program managers use ... more Increasing system complexity requires that engineers, systems analysts, and program managers use comprehensive design methodologies to deliver affordable and resilient designs. One method is set‐based design (SBD), a product development and managerial process distinctly suited for developing complex systems under uncertainty. SBD simultaneously develops, analyzes, and matures numerous potential design sets, enabling the identification of high‐value, affordable, and resilient designs. Published SBD research is a rich source of both qualitative and quantitative methods. This research specifically focuses on quantitative SBD methods to apply a value of information (VOI) methodology enabling design convergence and selection. We build upon previous SBD research to enable design maturation and uncertainty reduction. Our methodology integrates design maturation and multiobjective VOI analysis into a comprehensive quantitative SBD process to guide system development from initial design concepts to the pre‐production design decision. In doing so, we also provide refinements and process improvements to existing quantitative SBD methods. We demonstrate our methodology with a model‐based UAV design case study using an integrated suite of system, value, and cost models. Our case study specifically focuses on the design maturation and model selection decisions enabling design space convergence. We compare our current results with those from a previous UAV case study, achieving a 41% reduction in required computation time for design space convergence. These results highlight the methodology's ability to reduce program risk and potential to improve SBD convergence efficiency.
Systems Engineering, 2021
Increasing system complexity requires that engineers, systems analysts, and program managers use ... more Increasing system complexity requires that engineers, systems analysts, and program managers use comprehensive design methodologies to deliver affordable and resilient designs. One method is set-based design (SBD), a product development and managerial process distinctly suited for developing complex systems under uncertainty. SBD simultaneously develops, analyzes, and matures numerous potential design sets, enabling the identification of high-value, affordable, and resilient designs. Published SBD research is a rich source of both qualitative and quantitative methods. This research specifically focuses on quantitative SBD methods to apply a value of information (VOI) methodology enabling design convergence and selection. We build upon previous SBD research to enable design maturation and uncertainty reduction. Our methodology integrates design maturation and multiobjective VOI analysis into a comprehensive quantitative SBD process to guide system development from initial design concepts to the pre-production design decision. In doing so, we also provide refinements and process improvements to existing quantitative SBD methods. We demonstrate our methodology with a model-based UAV design case study using an integrated suite of system, value, and cost models. Our case study specifically focuses on the design maturation and model selection decisions enabling design space convergence. We compare our current results with those from a previous UAV case study, achieving a 41% reduction in required computation time for design space convergence. These results highlight the methodology's ability to reduce program risk and potential to improve SBD convergence efficiency.
Systems Engineering, 2021
Engineering complex systems is an exercise in sequential multiobjective decision making under unc... more Engineering complex systems is an exercise in sequential multiobjective decision making under uncertainty. One method for handling this complexity and uncertainty is set-based design (SBD). SBD is a concurrent engineering and management methodology that develops, analyzes, and matures numerous design options, reducing risk and delivering higher value to the stakeholders and end users. SBD accomplishes this through controlled design space convergence which reduces uncertainty and prevents premature design decisions. While SBD has been the subject of numerous scholarly articles, there is limited research providing quantitative methodologies that inform decisions enabling design maturation and convergence. We present a value of information (VOI) based methodology for multiobjective decision problems, and demonstrate its applicability for SBD decisions. We apply Bayesian decision models and information value to inform multiobjective modeling and design maturation decisions. Research contributions include: 1) a framework integrating VOI into the SBD process, 2) a multiobjective VOI method assessing a higher-resolution model's ability to reduce uncertainty, and 3) a means of informing modeling decisions by comparing multiple high resolutions models, given their usage cost and their potential to deliver information value. Finally, we demonstrate the inherent issues associated with premature decisions and traditional point-based design approaches which run the risk of selecting an alternative that later proves infeasible.
Systems Engineering, 2020
Increasing system complexity has provided the impetus to develop new and novel systems engineerin... more Increasing system complexity has provided the impetus to develop new and novel systems engineering methodologies. One of these methodologies is set-based design (SBD), a concurrent design methodology well suited for complex systems subject to significant uncertainty. Since the 1990s, numerous private, public, and defense sector design programs have successfully implemented SBD. However, concerns regarding SBD's complexity, tendency toward qualitative methods, and lack of quantitative tools have limited its use. To address these issues, our research surveys 122 refereed journal articles and conference papers to assess SBD's state-of-practice and identify relevant research opportunities. To accomplish these tasks, we perform a structured literature review to identify and assess relevant and influential research. We found that SBD's state-of-practice relies heavily upon decision and tradespace analysis with increasing emphasis on uncertainty modeling and MBSE. We found that the majority of SBD research consists of quantitative methodologies focusing on component and small system applications. We also found that complex system applications used mostly qualitative methodologies. We identify SBD research opportunities for requirements development, MBSE, uncertainty modeling, multiresolution modeling, adversarial analysis, and program management. Finally, we recommend the development of a comprehensive SBD methodology and toolkit, suited for complex system design across all stages of the product development life cycle.
IEEE Transactions on Engineering Management, 2021
System design is an exercise in sequential decisionmaking, with the objective of developing resil... more System design is an exercise in sequential decisionmaking, with the objective of developing resilient and affordable systems. Throughout the design process, engineering and program managers must balance several competing objectives, such as ensuring design feasibility, minimizing cost, schedule, and performance risk, while simultaneously achieving stakeholder value. Thus, engineering and program managers require design and analysis methods enabling complex and multiobjective design decisions under uncertainty. Unfortunately, there exists limited research providing quantitative methodologies specifically enabling program management decisions using quantitative set-based design. We, therefore, present a quantitative set maturation and uncertainty resolution decision methodology using value-focused multiobjective decision models and model-based engineering practices. This methodology assesses and quantifies uncertainty regarding stakeholder value, cost, requirements, and design maturity for each design set. These metrics facilitate the calculation of the program manager value, which when combined with design set feasibility entropy, enable tradeoff analysis informing design maturation and uncertainty resolution prioritization decisions. We develop and demonstrate our methodology using a model-based unmanned aerial vehicle case study implemented in the ModelCenter modeling environment. This methodology provides program managers an efficient, cost effective, and defensible approach to inform system design maturation and uncertainty resolution decisions enabling the development of resilient and affordable systems.
Applied Sciences, 2021
System designers, analysts, and engineers use various techniques to develop complex systems. A tr... more System designers, analysts, and engineers use various techniques to develop complex systems. A traditional design approach, point-based design (PBD), uses system decomposition and modeling, simulation, optimization, and analysis to find and compare discrete design alternatives. Set-based design (SBD) is a concurrent engineering technique that compares a large number of design alternatives grouped into sets. The existing SBD literature discusses the qualitative team based characteristics of SBD, but lacks insights into how to quantitatively perform SBD in a team environment. This paper proposes a qualitative SBD conceptual framework for system design, proposes a team-based, quantitative SBD approach for early system design and analysis, and uses an unmanned aerial vehicle case study with an integrated model-based engineering framework to demonstrate the potential benefits of SBD.We found that quantitative SBD tradespace exploration can identify potential designs, assess design feasibility, inform system requirement analysis, and evaluate feasible designs. Additionally, SBD helps designers and analysts assess design decisions by providing an understanding of how each design decision affects the feasible design space. We conclude that SBD provides a more holistic tradespace exploration process since it provides an integrated examination of system requirements and design decisions.
Developing nation-conflict forecasting models is of vital importance to international resource al... more Developing nation-conflict forecasting models is of vital importance to international resource allocation strategies that affect regional and worldwide security and stability. A nation in conflict may evolve very differently to one that is not. Using a unique set of open-source data, a suite of region-specific models are developed that predict nation-state transitions into and out of violent conflict. Conflict transition predictor variables differ not only between the regions, but also within each region dependent on if a nation is identified as in conflict or not. Accurate modeling of complex regional environments is achieved with parsimonious and interpretable models. While we ensure the causal variables of the previous study are considered for inclusion in the models developed, this paper focuses on the ability to accurately predict conflict over individual independent variable analysis.
Journal of Information Warfare, 2017
Modern military operations continue to be extraordinarily susceptible to the effects of cyber-bas... more Modern military operations continue to be extraordinarily susceptible to the effects of cyber-based Information Operations (IO). Within social media lies the ability to gain a clearer perspective of the 21st-century battlefields, enabling rapid and informed decision making and decisive action by commanders and their staffs. This paper discusses emerging trends, threats, and concepts that are being employed by numerous actors around the globe to gain positional advantage both internal and external to the cyberspace domain.
Thesis Chapters by Nick Shallcross
University of Arkansas, 2021
This dissertation comprises a body of research facilitating decision-making and complex system de... more This dissertation comprises a body of research facilitating decision-making and complex system development with quantitative set-based design (SBD). SBD is concurrent product development methodology, which develops and analyzes many design alternatives for longer time periods enabling design maturation and uncertainty reduction. SBD improves design space exploration, facilitating the identification of resilient and affordable systems. The literature contains numerous qualitative descriptions and quantitative methodologies describing limited aspects of the SBD process. However, there exist no methodologies enabling the quantitative management of SBD programs throughout the entire product development cycle. This research addresses this knowledge gap by developing the process framework and supporting methodologies guiding product development from initial system concepts to a final design solution. This research provides several new research contributions. First, we provide a comprehensive SBD state-of-practice assessment identifying key knowledge and methodology gaps. Second, we demonstrate the physical implementation of the integrated analytics framework in a model-based engineering environment. Third, we develop a quantitative methodology enabling program management decision making in SBD. Fourth, we describe a supporting uncertainty reduction methodology using multiobjective value of information analysis to assess design set maturity and higher-resolution model usefulness. Finally, we describe a quantitative SBD process framework enabling sequential design maturation and uncertainty reduction decisions. Using an unmanned aerial vehicle case study, we demonstrate our methodology’s ability to resolve uncertainty and converge a complex design space onto a set of resilient and affordable design solutions.
DTIC, 2016
The prediction and forecasting of violent conflict, is of vital importance to formulate coherent ... more The prediction and forecasting of violent conflict, is of vital importance to formulate coherent national strategies effecting regional and worldwide stability and security. Using open source data, this research formulates and constructs a suite of statistical models that predict future transitions into and out of violent conflict and forecasts the regional and global incidences of violent conflict over a ten-year time horizon. A total of thirty predictor variables are tested and evaluated for inclusion in twelve conditional logistic regression models, which calculate the probability that a nation will transition from its current conflict state, either “In Conflict” or “Not in Conflict”, to a new state in the following year. These probabilities are then used to construct a series of nation-specific Markov chain models that forecast violent conflict, as well as yield insights into regional conflict trends out to year 2024 and beyond. The logistic regression models proposed in this study achieve training dataset accuracies of 88.76%, and validation dataset accuracies of 84.67%. Additionally, the Markov models achieve three year forecast accuracies of 85.16% during model validation. Given the current state of included predictor variables, this study predicts that global violent conflict rates remain constant through year 2024, but are projected to increase beyond that timeframe with 95 of the 182 considered nations projected to be in a state of violent conflict from the current 84 nations in conflict.
Drafts by Nick Shallcross
The prediction and forecasting of violent conflict is of vital importance to formulation of inter... more The prediction and forecasting of violent conflict is of vital importance to formulation of international strategies effecting regional and worldwide security and stability. Using open source data, this paper discusses the formulation and construction of a suite of region-specific conditional logistic regression models that predict nation-state transitions into and out of violent conflict. We test and evaluate thirty open conflict predictor variables for inclusion in twelve conditional logistic regression models, which calculate the probability that a nation will transition from its current conflict state, either “In Conflict” or “Not in Conflict”, to a new state in the following year. Significant conflict transition predictor variables differ not only between the regions, but within each region depending on if a nation is identified as in conflict or not. This approach allows for the accurate modeling of complex regional environments with parsimonious and operationally interpretable models. The conditional logistic regression models proposed in this study achieve conflict transition prediction accuracies of 88.76% during model construction and conflict transition prediction accuracies of 84.67% model validation for 182 of the world’s nations. Model assessment using receiver operating characteristic curves, model accuracy, and the Hosmer-Lemeshow goodness of fit test demonstrate the models are statistically sound.
Uploads
Papers by Nick Shallcross
Thesis Chapters by Nick Shallcross
Drafts by Nick Shallcross