Maria Seale

Followers

Following

Public Views

Uploads

Papers by Maria Seale

Adverse Outcome Pathways for Engineered Systems

2023 IEEE International Systems Conference (SysCon)

Companies and organizations around the world spend massive amounts of money each year to discover... more Companies and organizations around the world spend massive amounts of money each year to discover, predict, and remediate failures within engineered systems. By doing so, these groups save lives, reduce costs, and maintain their reputations. However, to effectively and efficiently perform these tasks, companies and organizations rely on individuals with specialized knowledge in a variety of topics related to failure. This knowledge is often acquired through years of academic and onthe-job training centered around the review of scientific documentation such as books, reports, manuals, and peerreviewed publications. The loss of this knowledge through employee attrition can be detrimental to a group as knowledge is often difficult to reacquire. The aggregation and representation of known failure mechanisms for engineered materials could aid in the sharing of knowledge, the acquisition of knowledge, and the discovery of failure causes, reducing the risk of failure. Thus, the current work proposes the Adverse Outcome Pathway for Engineered Systems (AOP-ES) framework, an extension of the Adverse Outcome Pathway used in toxicology. The AOP-ES is designed to document failure knowledge enabling knowledge transfer and the prediction of failures of novel engineered materials based on the performance of similar materials. This paper introduces the AOP-ES framework and its key elements alongside the principles that govern the framework. An application of the framework is presented and additional benefits are explored.

Download

A Comparative Study of Continual, Lifelong, and Online Supervised Learning Libraries

The International FLAIRS Conference Proceedings

Machine learning has shown to be a crucial part of big data analytics; however, it lacks when the... more Machine learning has shown to be a crucial part of big data analytics; however, it lacks when the data is continuously streaming in from the system and changing too much from the original training data. Online learning is machine learning for streaming data that arrives in a sequential order where the model updates after every data point. While machine learning relies on well-established libraries such as PyTorch and Keras, the libraries for online learning are less well known, but they are here to serve similar purposes of reproducibility and reducing the time from research to production. Here, we compare different libraries for online learning research, specifically supervised learning. We compare them on the axes of developmental experience and benchmark testing as researchers. Our comparison as developers takes maintenance, documentation, and offerings of state-of-the-art algorithms into account. As this is not necessarily free of bias, we also use benchmarks known to online lea...

Download

An ontology for an epigenetics approach to prognostics and health management

Techniques in prognostics and health management have advanced considerably in the last few decade... more Techniques in prognostics and health management have advanced considerably in the last few decades, enabled by breakthroughs in computational methods and supporting technologies. These predictive models, whether data-driven or physics-based, target the modeling of a system’s aggregate performance. As such, they generalize assumptions about the modelled system’s components, and are thus limited in their ability to represent individual components and the dynamic environmental factors that affect composite system health. To address this deficiency, we have developed an epigenetics-inspired knowledge representation for engineered system state that encompasses components and environmental factors. Epigenetics is concerned with explaining how environmental factors affect the expression of an organism’s genetic material. The field has derived important in-sights into the development and progression of disease states based on how environmental factors impact genetic material, causing variat...

Download

Creating an Explainable Intrusion Detection System Using Self Organizing Maps

2022 IEEE Symposium Series on Computational Intelligence (SSCI)

Modern Artificial Intelligence (AI) enabled Intrusion Detection Systems (IDS) are complex black b... more Modern Artificial Intelligence (AI) enabled Intrusion Detection Systems (IDS) are complex black boxes. This means that a security analyst will have little to no explanation or clarification on why an IDS model made a particular prediction. A potential solution to this problem is to research and develop Explainable Intrusion Detection Systems (X-IDS) based on current capabilities in Explainable Artificial Intelligence (XAI). In this paper, we create a Self Organizing Maps (SOMs) based X-IDS system that is capable of producing explanatory visualizations. We leverage SOM's explainability to create both global and local explanations. An analyst can use global explanations to get a general idea of how a particular IDS model computes predictions. Local explanations are generated for individual datapoints to explain why a certain prediction value was computed. Furthermore, our SOM based X-IDS was evaluated on both explanation generation and traditional accuracy tests using the NSL-KDD and the CIC-IDS-2017 datasets.

Download

Scalability, Explainability and Performance of Data-Driven Algorithms in Predicting the Remaining Useful Life: A Comprehensive Review

This work summarizes the state-of-the-art data-driven methods for prediction of the Remaining Use... more This work summarizes the state-of-the-art data-driven methods for prediction of the Remaining Useful Life (RUL). It discusses challenges and open problems faced in PdM. This study presents a discussion on the new problems that need to be considered towards the Industry 4.0 goals. We propose the future direction for each challenge discussed in this article.

Download

Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunities

arXiv (Cornell University), Jul 13, 2022

The application of Artificial Intelligence (AI) and Machine Learning (ML) to cybersecurity challe... more The application of Artificial Intelligence (AI) and Machine Learning (ML) to cybersecurity challenges has gained traction in industry and academia, partially as a result of widespread malware attacks on critical systems such as cloud infrastructures and government institutions. Intrusion Detection Systems (IDS), using some forms of AI, have received widespread adoption due to their ability to handle vast amounts of data with a high prediction accuracy. These systems are hosted in the organizational Cyber Security Operation Center (CSoC) as a defense tool to monitor and detect malicious network flow that would otherwise impact the Confidentiality, Integrity, and Availability (CIA). CSoC analysts rely on these systems to make decisions about the detected threats. However, IDSs designed using Deep Learning (DL) techniques are often treated as black box models and do not provide a justification for their predictions. This creates a barrier for CSoC analysts, as they are unable to improve their decisions based on the model's predictions. One solution to this problem is to design explainable IDS (X-IDS). This survey reviews the state-of-the-art in explainable AI (XAI) for IDS, its current challenges, and discusses how these challenges span to the design of an X-IDS. In particular, we discuss black box and white box approaches comprehensively. We also present the tradeoff between these approaches in terms of their performance and ability to produce explanations. Furthermore, we propose a generic architecture that considers human-in-the-loop which can be used as a guideline when designing an X-IDS. Research recommendations are given from three critical viewpoints: the need to define explainability for IDS, the need to create explanations tailored to various stakeholders, and the need to design metrics to evaluate explanations.

Download

Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunities

IEEE Access

Download

Integrated Data Engineering for Automated Labeling (IDEAL) and Future Design of Aircraft

2019 IEEE Aerospace Conference

Data analysis and machine learning methods provide information that can be used to (1) reduce cos... more Data analysis and machine learning methods provide information that can be used to (1) reduce costs by enabling a condition-based maintenance program, (2) optimize maintenance scheduling, and (3) influence the designs of future platforms. The work presented here describes the integration of repurposed data and autonomous labeling of such engineered data compilations. This effort is called Integrated Data Engineering for Automated Labeling (IDEAL) and is based on an innovative data engineering approach called Data Programming. IDEAL aims to exploit the wealth of heterogeneous, historical data sets describing different aspects of aircraft performance from sources such as the aircraft onboard data collection system (Health and Usage Monitoring System/Flight Data Recorder) and the off-board maintenance logbook. By generating an integrated data landscape, IDEAL enables the exploration, exploitation, testing, and evaluation of innovative condition indicator (CI) algorithms for better representing the performance state of a platform and its components. In particular, IDEAL supports the Subject Matter Expert-guided characterization of ground-truth labels informing both the integration of disparate data sets and the classification of events within the resulting integrated data set. This mechanism results in massive labeled data sets that allow for the application of machine learning techniques to support the exposure of hidden phenomena embedded within platform performance data.

An Integrated Blockchain Approach for Provenance of Rotorcraft Maintenance Data

2020 IEEE Aerospace Conference

The U.S. Army Engineer Research and Development Center's Information Technology Laboratory (I... more The U.S. Army Engineer Research and Development Center's Information Technology Laboratory (ITL) is creating a Data Lake Ecosystem to support efficient storage, querying and analysis of terabyte-scale datasets within a High Performance Computing (HPC) environment. The datasets contain important current and historical data, and many datasets may be related to a single subject. The ecosystem must support the ability to perform holistic analysis across datasets and must ensure data integrity and security. For the latter, it is essential to keep a record of the creation and operations performed on a Data Lake object. This type of record-keeping is called data provenance. Assured data provenance can help detect the creation, manipulation, and deletion of objects within the Data Lake Ecosystem. However, developing assured data provenance remains a critical issue. There are challenges in secure collection and storage, verifiability, and privacy of the provenance data. Hence, there is a need for guaranteeing the security and integrity of data provenance for a Data Lake Ecosystem. In this paper, we propose a trusted data provenance application for rotorcraft maintenance data based on blockchain technology. Blockchain-based data provenance facilitates recording and tracking by treating the data as relevant assets in a transactional network. We designed and implemented a proof-of-concept application to collect and verify Data Lake provenance by embedding the data provenance on a private blockchain platform. This application allows the replication of data provenance on every node of a trusted closed network, ensuring high availability and fault tolerance. With the proposed blockchain model, data provenance for unique Data Lake objects can be stored securely and efficiently verified. Results from evaluations demonstrate that with blockchain technology, it is possible to provide secure, immutable, and reliable data provenance that is essential for maintaining the integrity of information in a Data Lake environment.

Similarity Based Methods for Faulty Pattern Detection in Predictive Maintenance

2021 International Conference on Computational Science and Computational Intelligence (CSCI)

A Dynamic Hyperbolic Surface Model for Responsive Data Mining

Procedia Computer Science

Abstract Data management systems impose structure on data via a static representation schema or d... more Abstract Data management systems impose structure on data via a static representation schema or data structure. Information from the data is extracted by executing queries based on predefined operators. This paradigm restricts the searchability of the data to concepts and relationships that are known or assumed to exist among the objects. While this is an effective and efficient means of retrieving simple information, we propose that such a structure severely limits the ability to derive breakthrough knowledge that exists in data under the guise of “unknown unknowns.” A dynamic system will alleviate this dependence, allowing theoretically infinite projections of the data to reveal discoverable relationships that are hidden by traditional use case-driven, static query systems. In this paper, we propose a framework for a data-responsive query algebra based on a dynamic hyperbolic surface model. Such a model could provide more intuitive access to analytics and insights from massive, aggregated datasets than existing methods. This model will significantly alter the means of addressing the underlying data by representing it as an arrangement on a dynamic, hyperbolic plane. Consequently, querying the data can be viewed as a process similar to quantum annealing, in terms of characterizing data representation as an energy minimization problem with numerous minima.

Deep Learning Approaches to Remaining Useful Life Prediction: A Survey

2021 IEEE Symposium Series on Computational Intelligence (SSCI)

Prognostic and Health Management (PHM) systems have multiple facets one would need to perfect for... more Prognostic and Health Management (PHM) systems have multiple facets one would need to perfect for an efficient system. One of these is the prediction of remaining useful life (RUL), which is the task of producing a number of time units (cycles, minutes, days, etc) until a part of the system or the system as a whole will fail. Over the years, deep learning approaches have been used to effectively perform this task, and these approaches fall into multiple different types of deep learning architectures. While non deep learning approaches exist, this paper focuses on a number of different deep learning approaches to solving the problem of RUL prediction.

Machinery Faults Prediction Using Ensemble Tree Classifiers: Bagging or Boosting?

Annual Conference of the PHM Society, 2021

One of the main goals of predictive maintenance is to accurately classify the temporal trends as ... more One of the main goals of predictive maintenance is to accurately classify the temporal trends as early as possible, detect faulty states and pinpoint the root cause of the fault. Undoubtedly, neither late nor early maintenance is desirable and incurs additional operating costs; however, early identification of faulty trends and scheduling on-time maintenance is crucial to smooth machinery operation. Various data-driven techniques have been used to identify faults; nevertheless, many of these techniques fail to perform when faced with missing values at run time or lack any explanation on the root cause of the fault. The present work offers a comprehensive study on different techniques used for fault type classification and compares their performance in identifying the mode of operation for the PHME21 dataset. We also evaluate the robustness of such classifiers against missing values. This study shows that tree-based techniques are best suited to perform root cause analysis for each f...

Download

An Approach to Gamifying Acquisitions for Assessing Impact on Military Strategy of Nation States

2021 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA), 2021

Complex Systems in which humans play a role, namely Human-Integrated Complex Systems (HICS), can ... more Complex Systems in which humans play a role, namely Human-Integrated Complex Systems (HICS), can be difficult to model or simulate due to the uncertainty introduced by the human component. Traditional modeling approaches such as physics-based modeling do not provide predictive insight towards situation awareness and management. War game designers, and game architects are familiar with HICS problem spaces, and use gamification of such complex contexts as a means of modeling human behavior to inform, predict, and manage an HICS style problem. The game play thereby becomes a means of providing situation awareness and management of the HICS by using human action during game play as a heuristic for pruning the intractable possibility space of the problem at large into a likely probability subspace based on the actions players actually take when playing an HICS game simulation. This paper explores the approach of gamification of real-world HICS problem spaces for situation awareness and management. A gamification methodology is introduced and investigated through the use case of military acquisitions.

A Survey of HMM-based Algorithms in Machinery Fault Prediction

2021 IEEE Symposium Series on Computational Intelligence (SSCI), 2021

Early detection of faulty patterns and timely scheduling of maintenance events can minimize risk ... more Early detection of faulty patterns and timely scheduling of maintenance events can minimize risk to the underlying processes and increase the system's lifespan, reliability, and availability. Different techniques are used in the literature to determine the health state of the system, one of which is the Hidden Markov Models (HMMs). This class of algorithms is very well suited for modeling the health condition dictated by the latent states of the system. HMMs can reveal transitions from one state to another, thus highlighting degradation in a system's health and the right time for maintenance. While many extensions and variations of the HMM are studied for a variety of applications, the present study aims to evaluate and compare the state-of-the-art HMM-based research in predictive maintenance only. This study also aims to discuss the capabilities and limitations of such algorithms and future directions to tackle the current limitations.

Data Lake Ecosystem Workflow

The Engineer Research and Development Center, Information Technology Laboratory’s (ERDC-ITL’s) Bi... more The Engineer Research and Development Center, Information Technology Laboratory’s (ERDC-ITL’s) Big Data Analytics team specializes in the analysis of large-scale datasets with capabilities across four research areas that require vast amounts of data to inform and drive analysis: large-scale data governance, deep learning and machine learning, natural language processing, and automated data labeling. Unfortunately, data transfer between government organizations is a complex and time-consuming process requiring coordination of multiple parties across multiple offices and organizations. Past successes in large-scale data analytics have placed a significant demand on ERDC-ITL researchers, highlighting that few individuals fully understand how to successfully transfer data between government organizations; future project success therefore depends on a small group of individuals to efficiently execute a complicated process. The Big Data Analytics team set out to develop a standardized wor...

Download

An Epistemological Model for a Data Analysis Process in Support of Verification and Validation

The verification and validation (V&V) of the data analysis process is critical for establishing t... more The verification and validation (V&V) of the data analysis process is critical for establishing the objective correctness of an analytic workflow. Yet, problems, mechanisms, and shortfalls for verifying and validating data analysis processes have not been investigated, understood, or well defined by the data analysis community. The processes of verification and validation evaluate the correctness of a logical mechanism, either computational or cognitive. Verification establishes whether the object of the evaluation performs as it was designed to perform. (“Does it do the thing right?”) Validation establishes whether the object of the evaluation performs accurately with respect to the real world. (“Does it do the right thing?”) Computational mechanisms producing numerical or statistical results are used by human analysts to gain an understanding about the real world from which the data came. The results of the computational mechanisms motivate cognitive associations that further driv...

An Epigenetic Modeling Approach for Adaptive Prognostics of Engineered Systems

Procedia Computer Science, 2021

Prognostics and health management (PHM) frameworks are widely used in engineered systems, such as... more Prognostics and health management (PHM) frameworks are widely used in engineered systems, such as manufacturing equipment, aircraft, and vehicles, to improve reliability, maintainability, and safety. Prognostic information for impending failures and remaining useful life is essential to inform decision-making by enabling cost versus risk estimates of maintenance actions. These estimates are generally provided by physics-based or data-driven models developed on historical information. Although current models provide some predictive capabilities, the ability to represent individualized dynamic factors that affect system health is limited. To address these shortcomings, we examine the biological phenomenon of epigenetics. Epigenetics provides insight into how environmental factors affect genetic expression in an organism, providing system health information that can be useful for predictions of future state. The means by which environmental factors influence epigenetic modifications leading to observable traits can be correlated to circumstances affecting system health. In this paper, we investigate the general parallels between the biological effects of epigenetic changes on cellular DNA to the influences leading to either system degradation and compromise, or improved system health. We also review a variety of epigenetic computational models and concepts, and present a general modeling framework to support adaptive system prognostics.

Download

Approaches for Using Machine Learning Algorithms with Large Label Sets for Rotorcraft Maintenance

2019 IEEE Aerospace Conference, 2019

The US Army Aviation and Missile Research, Development, and Engineering Center (AMRDEC), in colla... more The US Army Aviation and Missile Research, Development, and Engineering Center (AMRDEC), in collaboration with the US Army Engineer Research and Development Center (ERDC), is using machine learning (ML) to transform the way rotorcraft maintenance logbook event data is scored for reporting purposes. Traditionally, human analysts manually inspect data fields completed by maintenance personnel and, using published guidelines in conjunction with their personal expertise, provide sets of labels or scores for each event. These labels are stored with the data and provide valuable insight into maintenance event histories. However, the inequity between the enormous volume of maintenance data generated daily and the ability of analysts to score the data results in only 10% of all data receiving scores; therefore, 90% of the recorded data does not contain this important value added feature. Classification algorithms for automating this scoring process trained on existing labeled data sets have been implemented with promising results. A particularly challenging element of this problem, however, involves the classification of the specific component on which maintenance was performed. Greater than 1200 unique labels exist that can be used to describe a rotorcraft component that is the subject of a maintenance action. Furthermore, the component labels are hierarchically structured, resulting in the occurrence of multiple levels of precision in identifying a component in the expert-labeled data used for training. Although computational efficiency of common classification algorithms has improved considerably, it is still quite challenging to harness these methods for problems that include large numbers of unique class labels. This paper describes several novel strategies for solving this problem, based on hierarchical ensemble models and strategic label set segmentation. Through these approaches, a best overall total component classification accuracy of 96% was achieved, in conjunction with a total per record accuracy for three different label categories of 93%. The approaches implemented to handle the large label set for classification of rotorcraft components, along with classification performance measures, are discussed.

Leveraging Big Data to Discover High Yield Prognostics Applications

2021 IEEE International Conference on Prognostics and Health Management (ICPHM), 2021

A common scenario for many organizations in this information age entails volumes of data that are... more A common scenario for many organizations in this information age entails volumes of data that are easily collected and stored in repositories, but are not subsequently leveraged to improve products or processes due to a lack of resources, such as knowledgeable personnel or sufficient computing power. Rather than being collected as the result of a specific mandate for improvement, the data are collected simply because it is easy and inexpensive to do so, and because the data allegedly possess some, as of yet unknown, value. As the field of Prognostics and Health Management (PHM) grows, the research that is invested into finding the remaining useful life (RUL) of high value components can leverage these data repositories in many different ways. However, the task can seem overwhelming when there is no clear direction on how to use data to further specific goals. In this paper, we discuss the importance of PHM and provide a data-driven approach for discovering impactful uses of data for PHM challenges. This approach is illustrated through a research effort that explored a set of vehicle maintenance data and determined that analysis of vehicle generators would constitute the best initial return on investment.

Maria Seale

Uploads

Papers by Maria Seale

Log In