Hybrid Computational Models for Skill Acquisition

Annual Review in Automatic Programming, 1991

This essay is to point out some of the basic differences between conventional and real-time applications of AI and to provoke discussion on whether or not a new paradigm would facilitate better understanding of these differences. Some of the listed differences are illustrated by examples from references. An attempt has been made to project existing AI methods onto the deduction-induction axis so as to find an analogy with human behaviour under hard real-time constraints, e.g. reflexes. The conclusion is that the developers of real-time AI methods might benefit by using a "reflex based" paradigm. Kevwords. Reasoning under hard real-time constraints; deduction and induction based approaches; reflex baaed paradigm of reasoning

v Acknowledgements vii

Artificial Intelligence, 1995

Learning methods based on dynamic programming (DP) are receiving increasing attention in artificial intelligence. Researchers have argued that DP provides the appropriate basis for compiling planning results into reactive strategies for real-time control, as well as for learning such strategies when the system being controlled is incompletely known. We introduce an algorithm based on DP, which we call Real-Time DP (RTDP), by which an embedded system can improve its performance with experience. RTDP generalizes Korf's Learning-Real-Time-A* algorithm to problems involving uncertainty. We invoke results from the theory of asynchronous DP to prove that RTDP achieves optimal behavior in several different classes of problems. We also use the theory of asynchronous DP to illuminate aspects of other DP-based reinforcement learning methods such as Watkins' Q-Learning algorithm. A secondary aim of this article is to provide a bridge between AI research on real-time planning and learning and relevant concepts and algorithms from control theory.

International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 2023

In the modern era of rapid development in Deep Neural Networks, Reinforcement Learning (RL) has evolved into a pivotal and transformative technology. RL, a learning process where these machine agent interacts with several unknown environment through trial and error. The agent, responsive to the learning machine, go through these interaction, and start receiving feedback in the form of positive rewards or negative rewards like penalties from the environment, and constantly refines its behavior. This research paper offers an in-depth introduction to the foundational concepts of RL, focusing on Markov Decision Processes and various RL algorithms. Machine Learning (ML) is a subset of Artificial Intelligence, which deals with ‘‘the question of how to develop software agents (Machine) that improve automatically with experience’’. The basic three categories of Machine Learning are. 1. Supervised Learning 2. Unsupervised Learning 3. Reinforcement Learning RL method is that in any situation the agent has to choose between using its acquired knowledge of the environment i.e. using an action already tried or performed previously or exploring actions never tried before in that situation. In this review paper, we will discuss the most used learning algorithms in games robotics and healthcare, autonomous control as well as communication and networking, natural language processing.[1]

AAAI Spring Symposium on Challenges Requiring the Combination of Machine Learning and Knowledge Engineering, 2023

AI models for cybersecurity have to detect and defend against constantly evolving cyber threats. Much effort is spent building defenses for zero days and unseen variants of known cyber-attacks. Current AI models for cybersecurity struggle with these yet unseen threats due to the constantly evolving nature of threat vectors, vulnerabilities, and exploits. This paper shows that cybersecurity AI models will be improved and more general if we include semi-structured representations of background knowledge. This could include information about the software and systems, as well as information obtained from observing the behavior of malware samples captured and detonated in honeypots. We describe how we can transfer this knowledge into forms that the RL models can directly use for decision-making purposes.

Ai Magazine, 1988

AI Magazine Volume 9 Number 1 (1988) (© AAAI)

IEEE Canadian Conference on Electrical and Computer Engineering

The desire to make applications and machines more intelligent and the aspiration to enable their operation without human interaction have been driving innovations in neural networks, deep learning, and other machine learning techniques. Although reinforcement learning has been primarily used in video games, recent advancements and the development of diverse and powerful reinforcement algorithms have enabled the reinforcement learning community to move from playing video games to solving complex real-life problems in autonomous systems such as self-driving cars, delivery drones, and automated robotics. Understanding the environment of an application and the algorithms' limitations plays a vital role in selecting the appropriate reinforcement learning algorithm that successfully solves the problem on hand in an efficient manner. Consequently, in this study, we identify three main environment types and classify reinforcement learning algorithms according to those environment types. Moreover, within each category, we identify relationships between algorithms. The overview of each algorithm provides insight into the algorithms' foundations and reviews similarities and differences among algorithms. This study provides a perspective on the field and helps practitioners and researchers to select the appropriate algorithm for their use case.

arXiv (Cornell University), 2020

Penetration testing is a security exercise aimed at assessing the security of a system by simulating attacks against it. So far, penetration testing has been carried out mainly by trained human attackers and its success critically depended on the available expertise. Automating this practice constitutes a non-trivial problem, as the range of actions that a human expert may attempts against a system and the range of knowledge she relies on to take her decisions are hard to capture. In this paper, we focus our attention on simplified penetration testing problems expressed in the form of capture the flag hacking challenges, and we analyze how model-free reinforcement learning algorithms may help to solve them. In modeling these capture the flag competitions as reinforcement learning problems we highlight that a specific challenge that characterize penetration testing is the problem of discovering the structure of the problem at hand. We then show how this challenge may be eased by relying on different forms of prior knowledge that may be provided to the agent. In this way we demonstrate how the feasibility of tackling penetration testing using reinforcement learning may rest on a careful trade-off between model-free and model-based algorithms. By using techniques to inject a priori knowledge, we show it is possible to better direct the agent and restrict the space of its exploration problem, thus achieving solutions more efficiently.

2011

Abstract We describe a new framework for applying reinforcement learning (RL) algorithms to solve classification tasks by letting an agent act on the inputs and learn value functions. This paper describes how classification problems can be modeled using classification Markov decision processes and introduces the Max-Min ACLA algorithm, an extension of the novel RL algorithm called actor-critic learning automaton (ACLA).

… of the 8th international conference on …, 2007

Recently researchers have introduced methods to develop reusable knowledge in reinforcement learning (RL). In this paper, we define simple principles to combine skills in reinforcement learning. We present a skill combination method that uses trained skills to solve different tasks in a RL domain. Through this combination method, composite skills can be used to express tasks at a high level and they can also be re-used with different tasks in the context of the same problem domains. The method generates an abstract task representation based upon normal reinforcement learning which decreases the information coupling of states thus improving an agent's learning. The experimental results demonstrate that the skills combination method can effectively reduce the learning space, and so accelerate the learning speed of the RL agent. We also show in the examples that different tasks can be solved by combining simple reusable skills.