Designing and Evolving an Unreal TournamentTM 2004 Expert Bot

Journal of Computer Science and Technology, 2012

This paper investigates the performance and the results of an evolutionary algorithm (EA) specifically designed for evolving the decision engine of a program (which, in this context, is called bot) that plays Planet Wars. This game, which was chosen for the Google Artificial Intelligence Challenge in 2010, requires the bot to deal with multiple target planets, while achieving a certain degree of adaptability in order to defeat different opponents in different scenarios. The decision engine of the bot is initially based on a set of rules that have been defined after an empirical study, and a genetic algorithm (GA) is used for tuning the set of constants, weights and probabilities that those rules include, and therefore, the general behaviour of the bot. Then, the bot is supplied with the evolved decision engine and the results obtained when competing with other bots (a bot offered by Google as a sparring partner, and a scripted bot with a pre-established behaviour) are thoroughly analysed. The evaluation of the candidate solutions is based on the result of non-deterministic battles (and environmental interactions) against other bots, whose outcome depends on random draws as well as on the opponents' actions. Therefore, the proposed GA is dealing with a noisy fitness function. After analysing the effects of the noisy fitness, we conclude that tackling randomness via repeated combats and reevaluations reduces this effect and makes the GA a highly valuable approach for solving this problem.

ABSTRACT Defining human-like behavior for bots or non-player characters (NPC) in a real-time game such as Unreal Tournament 2004 (UT2004) has always been a challenging task. Dynamic game environments challenge the bot to behave autonomously without any human supervision or intervention. We highlight how we can use Behavior Trees to define agent‟ s behavior and use Reinforcement Learning (RL) technique to teach the bot how to choose actions so as to maximize numerical reward.

Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games, 2010

This paper presents an approach to the evolution of the cooperative behaviour of some bots inside the PC game Unreal TM. We intend to create bots that cooperate as a team trying to beat other teams (composed of human players or bots). So, in addition to the improvement of the default artificial intelligence (AI) of bots, we have performed an improvement of the 'team AI'. We have applied an evolutionary algorithm which optimizes the parameters considered in the hard-coded states inside the bot AI code, mainly those related to the cooperation. Two different approaches have been tested inside some different battle arenas: one considering a different set of parameters for every bot in the team, and the other one considering the same set of parameters for all the teammates. The results show that both methods yield better teams than the standard ones. The teams which share the same behaviour parameters, get a higher score than those with bots playing with different parameters.

This paper proposes an Evolutionary Algorithm for fine-tuning the behavior of a bot designed for playing Planet Wars, a game that has been selected for the the Google Artificial Intelligence Challenge 2010. The behavior engine of the proposed bot is based on a set of rules established by means of heuristic experimentation, followed by the application of an evolutionary algorithm to set the constants, weights and probabilities needed by those rules. This bot eventually defeated the baseline bot used to design it in most maps, and eventually played in the Google AI competition, obtaining a ranking in the top 20%.

Lecture Notes in Computer Science, 2011

This paper describes two different decision tree-based approaches to obtain strategies that control the behavior of bots in the context of the Unreal Tournament 2004. The first approach follows the traditional process existing in commercial videogames to program the game artificial intelligence (AI), that is to say, it consists of coding the strategy manually according to the AI programmer's experience with the aim of increasing player satisfaction. The second approach is based on evolutionary programming techniques and has the objective of automatically generating the game AI. An experimental analysis is conducted in order to evaluate the quality of our proposals. This analysis is executed on the basis of two fitness functions that were defined intuitively to provide entertainment to the player. Finally a comparison between the two approaches is done following the subjective evaluation principles imposed by the "2k bot prize" competition.

2015

Evolutionary Algorithms (EAs) are frequently used as a mechanism for the optimization of autonomous agents in games (bots), but knowing when to stop the evolution, when the bots are good enough, is not as easy as it would a priori seem. The first issue is that optimal bots are either unknown (and thus unusable as termination condition) or unreachable. In most EAs trying to find optimal bots fitness is evaluated through game playing. Many times it is found to be noisy, making its use as a termination condition also complicated. This paper summarizes our previous published work where we tested several termination conditions in order to find the one that yields optimal solutions within a restricted amount of time, to allow researchers to compare different EAs as fairly as possible. To achieve this, we examined several ways of finishing an EA who is finding an optimal bot design process for a particular game, Planet Wars in this case, with the characteristics described above, determining the capabilities of every one of them and, eventually, selecting one for future designs.

This paper illustrates how we create a software agent by employing FALCON, a self-organizing neural network that performs reinforcement learning, to play a well-known first person shooter computer game known as Unreal Tournament 2004. Through interacting with the game environment and its opponents, our agent learns in real-time without any human intervention. Our agent bot participated in the 2K Bot Prize competition, similar to the Turing test for intelligent agents, wherein human judges were tasked to identify whether their opponents in the game were human players or virtual agents. To perform well in the competition, an agent must act like human and be able to adapt to some changes made to the game. Although our agent did not emerge top in terms of humanlike, the overall performance of our agent was encouraging as it acquired the highest game score while staying convincing to be human-like in some judges' opinions.

2002

This paper describes and analyses a series of experiments intended to evolve a player for a variation of the classic arcade game Asteroids™ using steady state genetic programming. The player's behaviour is defined using a LISP like scripting language. While the game interprets scripts in real-time, such scripts are evolved off-line by a second program which simulates the realtime application. This method is used, as on-line evolution of the players would be too time consuming. A successful player needs to satisfy multiple conflicting objectives. This problem is addressed by the use of an automatically defined function (ADF) for each of these objectives in combination with task specific fitness functions. The overall fitness of evolved scripts is evaluated by a conventional fitness function. In addition to that, each of the ADFs is evaluated with a separate fitness function, tailored specifically to the objective that needs to be satisfied by that ADF.

GameBots is a project started in early 2000s by A. N. Marshall and G. A. Kaminka. The project aims at providing researchers a real-time virtual environment testbed for their agents. GameBots utilized environment of Unreal Tournament first-person shooter game providing several scenarios for the agents. GameBots project was continued by several research groups resulting in many interesting applications. In this paper we summarize evolution of the GameBots project and contributions made since the first appearance. We focus on Pogamut GameBots branch which has been steadily developed for six years with many improvements and optimizations.

Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games, 2010

Traditional approaches to game AI often feature behaviour that is scripted and predictable. Previous attempts at adaptive AI have struggled to get agents to learn quickly enough. This paper aims to show adaptation using online evolution is feasible and that it can can be incorporated with minimal change to the existing AI. A new approach is presented for evolving game agents online using an Evolution Strategy.