An Agent-to-Agent Framework for Concept Transfer

2011 11th International Conference on Intelligent Systems Design and Applications, 2011

This paper researches a collaborative strategy between an XCS learning classifier system (LCS) and a relational learning (RL) agent. The problem here is to learn a relational policy for a stochastic markovian decision process. In the proposed method the XCS agent is used to improve the performance of the RL agent by filtering the samples used at the induction step. This research shows that in these conditions, one of the main benefits of using the XCS algorithm comes from selecting the examples for relational learning using an estimation for the accuracy of the predicted value at each state-action pair. This kind of transfer learning is important because the characteristics of both agents are complementary: the RL agent incrementally induces a high level description of a policy, while the LCS agent offers adaptation to changes in the environment.

Lecture Notes in Computer Science, 2012

Transfer learning refers to the process of reusing knowledge from past tasks in order to speed up the learning procedure in new tasks. In reinforcement learning, where agents often require a considerable amount of training, transfer learning comprises a suitable solution for speeding up learning. Transfer learning methods have primarily been applied in single-agent reinforcement learning algorithms, while no prior work has addressed this issue in the case of multi-agent learning. This work proposes a novel method for transfer learning in multi-agent reinforcement learning domains. We test the proposed approach in a multiagent domain under various setups. The results demonstrate that the method helps to reduce the learning time and increase the asymptotic performance.

2008

Abstract Recent work in transfer learning has succeeded in making reinforcement learning algorithms more efficient by incorporating knowledge from previous tasks. However, such methods typically must be provided either a full model of the tasks or an explicit relation mapping one task into the other. An autonomous agent may not have access to such high-level information, but would be able to analyze its experience to find similarities between tasks.

2009

This article addresses collaborative learning in a multiagent system: each agent revises incrementally its beliefs B (a concept representation) to keep it consistent with the whole set of information K (the examples) that he has received from the environment or other agents. In SMILE this notion of consistency was extended to a group of agents and a unique consistent concept representation was so maintained inside the group. In the present paper, we present iSMILE in which the agents still provide examples to other agents but keep their own concept representation. We will see that iSMILE is more time consuming and loses part of its learning ability, but that when agents cooperate at classification time, the group benefits from the advantages of ensemble learning.

Lecture Notes in Computer Science, 2014

This paper presents an agent based approach to knowledge representation and learning methods. Agent architecture is described and discussed, together with its advantages and limitations. Main purpose of the proposed approach is to gain further insight in current teaching methods with a foremost aspiration for their improvement. Two different experimental studies were conducted; the first one addressing knowledge representation and the second one regarding knowledge transfer between agents. Obtained results are presented and analysed.

2008

Reinforcement learning agents typically require a significant amount of data before performing well on complex tasks. Transfer learning methods have made progress reducing sample complexity, but they have primarily been applied to model-free learning methods, not more data-efficient model-based learning methods. This paper introduces timbrel, a novel method capable of transferring information effectively into a model-based reinforcement learning algorithm.

Knowledge-Based Systems, 2007

Data mining has proven a successful gateway for discovering useful knowledge and for enhancing business intelligence in a range of 10 application fields. Incorporating this knowledge into already deployed applications, though, is highly impractical, since it requires recon-11 figurable software architectures, as well as human expert consulting. In an attempt to overcome this deficiency, we have developed Agent 12 Academy, an integrated development framework that supports both design and control of multi-agent systems (MAS), as well as ''agent 13 training''. We define agent training as the automated incorporation of logic structures generated through data mining into the agents of 14 the system. The increased flexibility and cooperation primitives of MAS, augmented with the training and retraining capabilities of Agent 15 Academy, provide a powerful means for the dynamic exploitation of data mining extracted knowledge. In this paper, we present the 16 methodology and tools for agent retraining. Through experimented results with the Agent Academy platform, we demonstrate how 17 the extracted knowledge can be formulated and how retraining can lead to the improvement -in the long run -of agent intelligence. 18

2008

A Relational Instance-Based Policy can be defined as an action policy described following a relational instancebased learning approach. The policy is represented with a set of state-goal-action tuples in some form of predicate logic and a distance metric: whenever the planner is in a state trying to reach a goal, the next action to execute is computed as the action associated to the closest state-goal pair in that set. In this work, the representation language is relational, following the ideas of Relational Reinforcement Learning. The policy to transfer (the set of state-goal-action tuples) is generated with a planning system solving optimally simple source problems. The target problems are defined in the same planning domain, have different initial and goal states to the source problems, and could be much more complex. We show that the transferred policy can solve similar problems to the ones used to learn it, but also more complex problems. In fact, the policy learned outperforms the planning system used to generate the initial state-action pairs in two ways: it is faster and scales up better.

Emergent neural computational …, 2001

We propose a theoretical framework for specification and analysis of a class of learning problems that arise in open-ended environments that contain multiple, distributed, dynamic data and knowledge sources. We introduce a family of learning operators for precise specification of some existing solutions and to facilitate the design and analysis of new algorithms for this class of problems. We state some properties of instance and hypothesis representations, and learning operators that make exact learning possible in some settings. We also explore some relationships between models of learning using different subsets of the proposed operators under certain assumptions.

2001

Multiagent systems offer a new paradigm where learning techniques can be useful. We focus on the application of lazy learning to multiagent systems where each agents learns individually and also learns when to cooperate in order to improve its performance. We show some experiments in which CBR agents use an adapted version of LID (Lazy Induction of Descriptions), a CBR method for classification. We discuss a collaboration policy (called Bounded Counsel) among agents that improves the agents’ performance with respect to their isolated performance. Later, we use decision tree induction and discretization techniques to learn how to tune the Bounded Counsel policy to a specific multiagent system—preserving always the individual autonomy of agents and the privacy of their case-bases. Empirical results concerning accuracy, cost, and robustness with respect to number of agents and case base size are presented. Moreover, comparisons with the Committee collaboration policy (where all agents collaborate always) are also presented.