Strategy games

Research Background: In Korea's amateur esports scene, a phenomenon is frequently observed where teams with individually skilled players experience consecutive defeats in team-based matches despite high individual performance levels. This phenomenon is presumed to stem from structural discrepancies between individual skills and team strategic execution capabilities. Research Objective: This study aims to systematically analyze the repetitive failure factors occurring in scrimmages (practice matches) for amateur teams preparing for the 4th Jeju Esports Challenge 2025. The goal is to identify failure mechanisms focusing on tactical decision-making processes, team communication systems, and individual players' role performance. Research Method: A mixed-method approach was applied, combining replay analysis, voice chat content analysis, and in-depth interviews for 15 scrimmage matches of a 5-person team composed of Master-Diamond tier players. In-game data was analyzed in minute-by-minute segments, with comparative analysis conducted against professional league matches with similar compositions. Key Results: The study identified four major failure factors: First, loss of laning phase dominance due to failed situational item build optimization. For example, top laner Renekton failed to respond to opponent Gwen's sustained magic damage, leading to mid-game laning disadvantage. Second, game tempo loss due to objective priority setting errors was observed. Notably, at the 6-minute dragon timing, the jungler prioritized scuttle crab, resulting in the first dragon being conceded to the opposing team. Third, synchronization failure in ultimate ability combo systems emerged as a core issue. Despite the team composition being designed around a 3-stage ultimate combo of Galio-Zyra-Renata, actual team fights achieved only a 40% success rate. Finally, inefficiency in team communication systems was confirmed, with emotional communication comprising 30% of total voice chat, interfering with crucial information delivery. Conclusion and Implications: This study identified structural causes of difficulties experienced by amateur teams in team-unit performance separate from individual skills. It confirmed the necessity for systematic team training programs and the importance of datadriven strategic analysis. The research results are expected to provide practical improvement directions for other amateur teams facing similar situations.

Improving the clarity of games allows players to spend more of their mental effort on strategic planning rather than the mundane bookkeeping of calculating legal moves. This article discusses techniques for achieving this, by making the design do the work rather than the player, and demonstrates this concept through example. Such techniques include visual design, simplifying rules, clarifying rules, harnessing emergent strategies, and minimising mental bookkeeping.

The work deals with the relationship between game mechanics and user experience. Identifies the basic gameplay mechanics and elements that have the most important influence on the quality of user experience. Research is realized on the example of a specific game using the concept of game design patterns. Part of the work deals with the introduction of this concept. Furthermore, the work deals with the general theories of games, models of classification and history of computer games. Part of the work is a description of the author's practical experience in creating games as a school project.Práce se zabývá vztahem herních mechanismů a uživatelského prožitku. Identifikuje základní herní mechanismy a prvky, které mají nejdůležitější vliv na kvalitu uživatelského prožitku. Výzkum je realizován na příkladu konkrétní hry pomocí konceptu návrhových vzorů. Část práce se zabývá představením tohoto konceptu. Dále se práce zabývá obecnými teoriemi her, modely klasifikace a historií počítač...

Mimicry and play are fundamental learning processes by which individuals can acquire behaviours, skills and norms. In this paper we utilise these two processes to create new game characters by mimicking and learning from actual human players. We present our approach towards aiding the design process of game characters through the use of genetic programming. The current state of the art in game character design relies heavily on human designers to manually create and edit scripts and rules for game characters. Computational creativity approaches this issue with fully autonomous character generators, replacing most of the design process using black box solutions such as neural networks. Our GP approach to this problem not only mimics actual human play but creates character controllers which can be further authored and developed by a designer. This keeps the designer in the loop while reducing repetitive labour. Our system also provides insights into how players express themselves in games and into deriving appropriate models for representing those insights. We present our framework and preliminary results supporting our claim.

This paper describes a co-evolutionary algorithm for generating simple spatially oriented tactics and considers whether students can learn better by playing against co-evolved opponents or by playing against an expert system or other similar hard-coded opponent. Although a number of artificially intelligent tutoring and e-learning systems exist, our work looks at using co-evolution to generate competent opponents for human students to learn from. This paper describes and discusses early results on using computationally intelligent opponents for tactical training of human students. Initial results indicate that the learning environment for human players, measured by game difficulty and transfer of training, is comparable across coevolved and hard-coded computer opponents.

Real-Time Strategy (RTS) games have become an attractive domain for AI research in recent years, due to their dynamic, multi-agent and multi-objective environments. Micromanagement, a core component of many RTS games, involves the control of multiple agents to accomplish goals that require fast, real time assessment and reaction. In this paper, we present the application and evaluation of a Neuroevolution technique in evolving micromanagement agents for the RTS game Starcraft: Brood War (SC:BW). The NeuroEvolution of Augmented Topologies (NEAT) algorithm, both in its standard form and its real-time variant (rtNEAT) is comparatively evaluated in micromanagement tasks. Preliminary results suggest the general viability of these techniques in comparison to traditional, non-adaptive AI. Further analysis of each algorithm identified differences in task performance and learning rate.

We present an approach to procedurally generate the narrative of a simple murder mystery. As a basis for the simulation, we use a rule evaluation system inspired by Ceptre, which employs linear logic to resolve valid actions during each step of the simulation. We extend Ceptre’s system with a concept of believable agents to make consecutive actions appear to have a causal connection so that players can comprehend the flow of events. The parts of the generated narratives are then presented to a player whose task it is to figure out who the murderer in this story could have been. Rather than aiming to replace highly authored narratives, this project generates puzzles, which may contain emerging arcs of a story as perceived by the player. While we found that even a simple rule set can create stories that are interesting to reason about, we expect that this type of system is flexible enough to create considerably more engaging stories if enough time is invested in authoring more complex...

Bien souvent, les jeux de stratégie proposent une vision du pouvoir qui a peu de nuances, où le/la joueur/se est toute-puissante dans une relation de domination. Or, l'expression du pouvoir dans certains jeux nous semble relever d'un ethos plus nuancé. Des jeux vidéo singuliers permettent, voire favorisent, une autre vision du monde, plus subtile et moins hégémonique. En ce sens, cet article propose trois analyses de jeux de stratégie où le pouvoir est complexe, instable ou déconstruit pour des raisons différentes. Ainsi, nous explorons trois figures hétérogènes à travers ces analyses : la menace fantôme dans X-COM: UFO Defense (Mythos Games, 1994), la personnification dans Ogre Battle (Enix, 1993) et la toile d'alliances dans Europa Universalis IV (Paradox Development Studio, 2013). Nous soutenons que des figures alternatives du jeu vidéo de stratégie peuvent subvertir la tendance dominante du genre pour permettre une réflexion critique sur celuici. Ces figures singulières soulignent la diversité des représentations et des visions du monde d'un corpus de jeux qui gagnerait à être exploré davantage.

Bien souvent, les jeux de stratégie proposent une vision du pouvoir qui a peu de nuances, où le/la joueur/se est toute-puissante dans une relation de domination. Or, l'expression du pouvoir dans certains jeux nous semble relever d'un ethos plus nuancé. Des jeux vidéo singuliers permettent, voire favorisent, une autre vision du monde, plus subtile et moins hégémonique. En ce sens, cet article propose trois analyses de jeux de stratégie où le pouvoir est complexe, instable ou déconstruit pour des raisons différentes. Ainsi, nous explorons trois figures hétérogènes à travers ces analyses : la menace fantôme dans X-COM: UFO Defense (Mythos Games, 1994), la personnification dans Ogre Battle (Enix, 1993) et la toile d'alliances dans Europa Universalis IV (Paradox Development Studio, 2013). Nous soutenons que des figures alternatives du jeu vidéo de stratégie peuvent subvertir la tendance dominante du genre pour permettre une réflexion critique sur celuici. Ces figures singulières soulignent la diversité des représentations et des visions du monde d'un corpus de jeux qui gagnerait à être exploré davantage.

Monte Carlo Tree Search methods provide a general framework for modeling decision problems by randomly sampling the decision space and constructing a search tree according to the sampling results. Artificial Intelligences employing these methods in games with massive decision spaces such as Go and Settlers of Cataan have recently demonstrated far superior results compared to the previous classic game theory approaches [Browne et al. 1]. We apply Monte Carlo Tree Search methods, particularly the UCT variant, to an online version of the popular board game Risk.

Although games and ideology may initially seem completely different, their relationship has become socially controversial due to their structural similarities. This debate arises from the fact that both games and ideologies function as information systems. The aim of this research is to open up the social discussion on the relationship between games, ideology, and communication systems. It seeks to examine the ideological purpose of information technologies that mimic communication actions within the framework of game theory. The development of game theory, its interaction with social sciences, and the relationship between games and communication systems have been analysed through the method of historical materialism. According to the data obtained, 1) the convergence between communication systems and game theory has been consciously designed to serve the objectives of the dominant ideology. Its aim is to manage societal concerns and fears by controlling them through a single system. 2) Communication systems, which allow for the continuous monitoring of societal needs, have been created through the combination of evolutionary theory, special relativity theory, and the theory of communicative action. 3) The mechanism created by this convergence generates the sustainability of interdisciplinary convergence, information technologies that produce digitized knowledge, and the invisibility of all these processes to the user. 4) The merging of concrete (communication systems) and abstract (human communication action) information systems with game theory has facilitated the emergence of new communication technologies and provided knowledge for social governance. 5) As a result, through these two merging systems, users are prevented from seeing the aims and goals of the action possibilities offered by these technologies. Thus, the continuous reproduction of the system encourages individuals to work unconsciously.

Reinforcement Learning (RL) gained a huge amount of popularity in computer science; applied in fields such as gaming, intelligent robots, remote sensing, and so on. The objective of reinforcement learning is to generate the optimal policy. The main problem of that optimal policy is that it is not fully guaranteed to be satisfied all the system specifications. Model checking is a technique to verify the system to meet the system specifications. PRISM-games is one of the model-checking tools that is used to verify the probabilistic system with competitive or collaborative behavior. Safe Reinforcement Learning via Shielding is a method that uses shield to restrict the action of the RL agent if it violates the specification using temporal logic. This paper presents to compare the winning strategies between three agents; Monte-Carlo Tree Search agent (MCTS), RL agent and shielded RL agent (SRL) which uses PRISM-games to restrict the action based on Tic-Tac-Toe game. Over thousand times of simulations has been made, the experiments show that MCTS agent has the highest win rate compared to other agents, but the losing rate of the shielded agent is reduced by using PRISM-games.

In this work, we propose a Dynamic Difficulty Adjustment methodology to achieve automatic video game balance. The balance task is modeled as a meta game, a game where actions change the rules of another base game. Based on the model of Reinforcement Learning (RL), an agent assumes the role of a game master and learns its optimal policy by playing the meta game. In this new methodology we extend traditional RL by adding the existence of a meta environment whose state transition depends on the evolution of a base environment. In addition, we propose a Multi Agent System training model for the game master agent, where it plays against multiple agent opponents, each with a distinct behavior and proficiency level while playing the base game. Our experiment is conducted on an adaptive grid-world environment in singleplayer and multiplayer scenarios. Our results are expressed in twofold: (i) the resulting decision making by the game master through gameplay, which must comply in accordance to an established balance objective by the game designer; (ii) the initial conception of a framework for automatic game balance, where the balance task design is reduced to the modulation of a reward function (balance reward), an action space (balance strategies) and the definition of a balance space state.

Although utilising computers to play board games has been a topic of research for many decades, the recent rapid developments in the field of reinforcement learning-like AlphaZero (and variants)-brought unprecedented progress in games such as chess and Go. However, the efficiency of this process remains unknown. In this work, we analyse the cost and efficiency of the AlphaZero approach when building a new game engine. Thus, we present our experience building AZ-Hive, an AlphaZero-based playing engine for the game of Hive. Using only the rules of the game and a quality of play assessment, AZ-Hive learns to play the game from scratch. Getting AZ-Hive up and running requires encoding the game in AlphaZero, i.e., capturing the board, the game state, the rules and the assessment of play-quality. And different encodings lead to significantly different AZ-Hive engines, with very different performance results. Thus, we propose a design space for configuring AZ-Hive, and demonstrate the costs and benefits of different configurations in this space. We find that different configurations lead to a less or more competitive playing-engine, but the training and evaluation for different such engines is prohibitively expensive. Moreover, no systematic, efficient exploration or pruning of the space is possible. In turn, an exhaustive exploration can easily take tens of training-years. CCS Concepts • Computer systems organization → Multicore architectures; • Software and its engineering → Software performance; • Computing methodologies → Parallel programming languages; Massively parallel algorithms; Artificial intelligence.

Aljabar linier merupakan salah satu cabang matematika yang mempelajari hubungan linier antara himpunan objek. Konsep-konsep dalam aljabar linier sering kali dapat diterapkan dalam berbagai bidang, termasuk dalam analisis dan strategi permainan. Dalam konteks permainan, aljabar linier digunakan untuk memodelkan dan menganalisis strategi yang digunakan oleh pemain.

AI techniques are already widely used in game software to provide computer-controlled opponents for human players. However, game design is a more-challenging problem than game play. Designers typically expend great effort to ensure that their games are balanced and challenging. Dynamic game-balancing techniques have been developed to modify a game-engine’s parameters in response to user play. In this paper we describe a first attempt at using AI techniques to design balanced board games like checkers and Go by modifying the rules of the game, not just the rule parameters. Our approach involves the use of a commercial general game-playing (GGP) engine that plays according to rules that are specified in a general game-definition language. We use a genetic algorithm (GA) to search the space of game rules, looking for turn-based board games that are well balanced, i.e., those that the GGP engine in self-play finds equally hard to win from either side and rarely draws. The GA finds bette...

In this work, we presented a study regarding two important aspects of evolving feature-based game evaluation functions: the choice of genome representation and the choice of opponent used to test the model. We compared three representations. One simpler and more limited, based on a vector of weights that are used in a linear combination of predefined game features. And two more complex, based on binary and n-ary trees. On top of this test, we also investigated the influence of fitness defined as a simulation-based function that: plays against a fixed weak opponent, plays against a fixed strong opponent, and plays against the best individual from the previous population. For a testbed, we have chosen a recently popular domain of digital collectible card games. We encoded our experiments in a programming game, Legends of Code and Magic, used in Strategy Card Game AI Competition. However, as the problems stated are of general nature we are convinced that our observations are applicable in the other domains as well.

We deal with the problem of automatic generation of complete rules of an arbitrary game. This requires a generic and accurate evaluating function that is used to score games. Recently, the idea that game quality can be measured using differences in performance of various game-playing algorithms of different strengths has been proposed; this is called Relative Algorithm Performance Profiles. We formalize this method into a generally application algorithm estimating game quality, according to some set of model games with properties that we want to reproduce. We applied our method to evolve chess-like boardgames. The results show that we can obtain playable and balanced games of high quality.

This research was conducted by an interdisciplinary team of two undergraduate students and a faculty to explore solutions to the Birds of a Feather (BoF) Research Challenge. BoF is a newly-designed perfect-information solitaire-type game. The focus of the study was to design and implement different algorithms and evaluate their effectiveness. The team compared the provided depth-first search (DFS) to heuristic algorithms such as Monte Carlo tree search (MCTS), as well as a novel heuristic search algorithm guided by machine learning. Since all of the studied algorithms converge to a solution from a solvable deal, effectiveness of each approach was measured by how quickly a solution was reached, and how many nodes were traversed until a solution was reached. The employed methods have a potential to provide artificial intelligence enthusiasts with a better understanding of BoF and novel ways to solve perfect-information games and puzzles in general. The results indicate that the propos...

Monte Carlo Tree Search (MCTS) is a sampling best-first method to search for optimal decisions. The success of MCTS depends heavily on how the tree is built and the selection process plays a fundamental role in this. One particular selection mechanism that has proved to be reliable is based on the Upper Confidence Bounds for Trees (UCT). The UCT attempts to balance exploration and exploitation by considering the values stored in the statistical tree of the MCTS. However, some tuning of the MCTS UCT is necessary for this to work well. In this work, we use Evolutionary Algorithms (EAs) to evolve mathematical expressions with the goal to substitute the UCT formula and use the evolved expressions in MCTS. More specifically, we evolve expressions by means of our proposed Semantic-inspired Evolutionary Algorithm in MCTS approach (SIEA-MCTS). This is inspired by semantics in Genetic Programming (GP), where the use of fitness cases is seen as a requirement to be adopted in GP. Fitness cases are normally used to determine the fitness of individuals and can be used to compute the semantic similarity (or dissimilarity) of individuals. However, fitness cases are not available in MCTS. We extend this notion by using multiple reward values from MCTS that allow us to determine both the fitness of an individual and its semantics. By doing so, we show how SIEA-MCTS is able to successfully evolve mathematical expressions that yield better or competitive results compared to UCT without the need of tuning these evolved expressions. We compare the performance of the proposed SIEA-MCTS against MCTS algorithms, MCTS Rapid Action Value Estimation algorithms, three variants of the *-minimax family of algorithms, a random controller and two more EA approaches. We consistently show how SIEA-MCTS outperforms most of these intelligent controllers in the challenging game of Carcassonne, whose statespace complexity is, approx., 10 40 .

Monte Carlo Tree Search (MCTS) is a relatively new sampling method with multiple variants in the literature. They can be applied to a wide variety of challenging domains including board games, video games, and energy-based problems to mention a few. In this work, we explore the use of the vanilla MCTS and the MCTS with Rapid Action Value Estimation (MCTS-RAVE) in the game of Carcassonne, a stochastic game with a deceptive scoring system where limited research has been conducted. We compare the strengths of the MCTS-based methods with the Star2.5 algorithm, previously reported to yield competitive results in the game of Carcassonne when a domainspecific heuristic is used to evaluate the game states. We analyse the particularities of the strategies adopted by the algorithms when they share a common reward system. The MCTS-based methods consistently outperformed the Star2.5 algorithm given their ability to find and follow long-term strategies, with the vanilla MCTS exhibiting a more robust game-play than the MCTS-RAVE.

Search-based procedural content generation uses stochastic global optimization algorithms to search for game content. However, standard tree search algorithms can be competitive with evolution on some optimization problems. We investigate the applicability of several tree search methods to level generation and compare them systematically with several optimization algorithms, including evolutionary algorithms. We compare them on three different game level generation problems: Binary, Zelda, and Sokoban. We introduce two new representations that can help tree search algorithms deal with the large branching factor of the generation problem. We find that in general, optimization algorithms clearly outperform tree search algorithms, but given the right problem representation certain tree search algorithms performs similarly to optimization algorithms, and in one particular problem, we see surprisingly strong results from MCTS.

Search-based procedural content generation uses stochastic global optimization algorithms to search for game content. However, standard tree search algorithms can be competitive with evolution on some optimization problems. We investigate the applicability of several tree search methods to level generation and compare them systematically with several optimization algorithms, including evolutionary algorithms. We compare them on three different game level generation problems: Binary, Zelda, and Sokoban. We introduce two new representations that can help tree search algorithms deal with the large branching factor of the generation problem. We find that in general, optimization algorithms clearly outperform tree search algorithms, but given the right problem representation certain tree search algorithms performs similarly to optimization algorithms, and in one particular problem, we see surprisingly strong results from MCTS.

We investigate how reinforcement learning can be used to train level-designing agents. This represents a new approach to procedural content generation in games, where level design is framed as a game, and the content generator itself is learned. By seeing the design problem as a sequential task, we can use reinforcement learning to learn how to take the next action so that the expected final level quality is maximized. This approach can be used when few or no examples exist to train from, and the trained generator is very fast. We investigate three different ways of transforming two-dimensional level design problems into Markov decision processes, and apply these to three game environments.

We present a new concept called Game Mechanic Alignment theory as a way to organize game mechanics through the lens of systemic rewards and agential motivations. By disentangling player and systemic influences, mechanics may be better identified for use in an automated tutorial generation system, which could tailor tutorials for a particular playstyle or player. Within, we apply this theory to several well-known games to demonstrate how designers can benefit from it, we describe a methodology for how to estimate "mechanic alignment", and we apply this methodology on multiple games in the GVGAI framework. We discuss how effectively this estimation captures agential motivations and systemic rewards and how our theory could be used as an alternative way to find mechanics for tutorial generation. CCS CONCEPTS • Applied computing → Computer games; • Human-centered computing → Information visualization; • Mathematics of computing → Nonparametric statistics.

We present a new method of automatic critical mechanic discovery for video games using a combination of game description parsing and playtrace information. This method is applied to several games within the General Video Game Artificial Intelligence (GVG-AI) framework. In a user study, human-identified mechanics are compared against system-identified critical mechanics to verify alignment between humans and the system. The results of the study demonstrate that the new method is able to match humans with higher consistency than baseline. Our system is further validated by comparing MCTS agents augmented with critical mechanics and vanilla MCTS agents on 4 games from GVG-AI. Our new playtrace method shows a significant performance improvement over the baseline for all 4 tested games. The proposed method also shows either matched or improved performance over the old method, demonstrating that playtrace information is responsible for more complete critical mechanic discovery. CCS CONCEPTS • Applied computing → Computer games; • Computing methodologies → Game tree search.

Human chess players prefer training with human opponents over chess agents as the latter are distinctively different in level and style than humans. Chess agents designed for human-agent play are capable of adjusting their level, however their style is not aligned with that of human players. In this paper, we propose a novel approach for designing such agents by integrating the theory of chess players' decision-making with a state-of-the-art Monte Carlo Tree Search (MCTS) algorithm. We demonstrate the benefits of our approach using two sets of analyses. Quantitatively, we establish that the agents attain their desired Elo ratings. Qualitatively, through a Turing-inspired test with a human chess expert, we show that our agents are indistinguishable from human players. CCS CONCEPTS • Computing methodologies → Artificial intelligence; • Applied computing → Computer games.

This paper presents a synthetic approach for generating role playing simulation games intended to support analysts (and trainees) interested in testing alternative competing courses of action (operations) and discovering what effects they are likely to precipitate in potential ethno-political conflict situations. Simulated leaders and followers capable of playing these games are implemented in a cognitive modeling framework, called PMFserv, which covers value systems, personality and cultural factors, emotions, relationships, perception, stress/coping style and decision making. Of direct interest, as Sect. 1.1 explains, is mathematical representation and synthesis of best-of-breed behavioral science models within this framework to reduce dimensionality and to improve the realism and internal validity of the agent implementations. Sections 2 and 3 present this for leader profiling instruments and group membership decision-making, respectively. Section 4 serves as an existence proof t...

This paper investigates the applicability of Genetic Programming type systems to dynamic game environments. Grammatical Evolution was used to evolve Behaviour Trees, in order to create controllers for the Mario AI Benchmark. The results obtained reinforce the applicability of evolutionary programming systems to the development of artificial intelligence in games, and in dynamic systems in general, illustrating their viability as an alternative to more standard AI techniques.

This paper presents a method for generating complex problems that allow multiple non-obvious solutions for the Physical Travelling Salesman Problem (PTSP). PTSP is a single-player game adaptation of the classical Travelling Salesman Problem that makes use of a simple physics model: the player has to visit a number of waypoints as quickly as possible by navigating a ship in real time across an obstaclefilled two-dimensional map. The difficulty of this game depends on the distribution of waypoints and obstacles across the twodimensional plane. Due to the physics of the game, the shortest route is not necessarily the fastest, as the ship's momentum makes it difficult to turn sharply at speed. This paper proposes an evolutionary approach to obtaining maps where the optimal solution is not immediately obvious. In particular, any optimal route for these maps should differ distinctively from (a) the optimal distance-based TSP route and (b) the route that corresponds to always approaching the nearest waypoint first. To achieve this, the evolutionary algorithm CMA-ES is employed, where maps, indirectly represented as vectors of real numbers, are evolved to differentiate maximally between a game-playing agent that follows two or more different routes. The results presented in this paper show that CMA-ES is able to generate maps that fulfil the desired conditions.

Designing a robust and adaptable Artificial Intelligence (AI) opponent in a computer game would ensure the game continues to challenge, immerse and excite the players at any stage. The outcomes of card based games such as "Heartstone: Heroes of Warcraft", aside the player skills, heavily depend on the initial composition of player card decks. To evaluate this impact we have developed a new robust regression network in a context of the AAIA Data Mining Competition 2018, which tries to predict the average win-rates of the specific combinations of bot-player and card decks. Our network is composed of 2 levels: the entry level with an array of finely optimized state of the art regression models including Extreme Learning Machines (ELM), Extreme Gradient Boosted decision tree (XGBOOST), and Least Absolute Shrinkage and Selection Operator (LASSO) regression trained via supervised learning on the labeled training dataset; and just a single ELM at the 2 nd level installed to learn to correct the predictions from the 1 st level. The final solution received the root of the mean squared error (RMSE) of just 5.65% and scored the 2 nd place in AAIA'2018 competition. This paper also presents two other runner-up models receiving RMSE of 5.7% and 5.86%, scoring the 4 th and the 6 th place respectively.

This paper describes an automated evaluation of the overall game experience using a synthetic agent, that we contextualize for First-Person Shooter games. This evaluation method is based on the characterization of the game experience through dynamics of major FPS games. We define dynamics as sequences of events that are meaningful for the player during the game session. As they trigger players' emotional responses, and influence their overall enjoyment and motivation, we classify them according to Motives for Play like curiosity, thrill-seeking, problem-solving, victory, and acquisition, in order to facilitate the evaluation process. Based on that, our evaluation method proposes to select synthetic agent routines that target a distinct game experience while playing a game session, using a selection of game dynamics. As the agent navigates through the level and interacts with opponents, dynamics may occur and, if so, are automatically identified, and then classified as Motives for Play. In the end, this classification can be used to evaluate the game experience and the quality of the level itself during playtesting sessions. It may also be utilized to assist the procedural generation of any level that target a specific game experience. CCS Concepts: • Human-centered computing → User models; • Applied computing → Computer games; Psychology.

Video games are complex simulation environments with many real-world properties that need to be addressed in order to build robust intelligence. In particular, realtime strategy games provide a multi-scale challenge which requires both deliberative and reactive reasoning processes. Experts approach this task by studying a corpus of games, building models for anticipating opponent actions, and practicing within the game environment. We motivate the need for integrating heterogeneous approaches by enumerating a range of competencies involved in gameplay and discuss how they are being implemented in EISBot, a reactive planning agent that we have applied to the task of playing real-time strategy games at the same granularity as humans.

The design of algorithms for Game AI agents usually focuses on the single objective of winning, or maximizing a given score. Even if the heuristic that guides the search (for reinforcement learning or evolutionary approaches) is composed of several factors, these typically provide a single numeric value (reward or fitness, respectively) to be optimized. Multi-Objective approaches are an alternative concept to face these problems, as they try to optimize several objectives, often contradictory, at the same time. This paper proposes for the first time a study of Multi-Objective approaches for General Video Game playing, where the game to be played is not known a priori by the agent. The experimental study described here compares several algorithms in this setting, and the results suggest that Multi-Objective approaches can perform even better than their single-objective counterparts.

Ms Pac-Man is one of the most challenging test beds in game artificial intelligence (AI). Genetic programming and Monte Carlo Tree Search (MCTS) have already been successful applied to several games including Pac-Man. In this paper, we use Monte Carlo Tree Search to create a Ms Pac-Man playing agent before using genetic programming to enhance its performance by evolving a new default policy to replace the random agent used in the simulations. The new agent with the evolved default policy was able to achieve an 18% increase on its average score over the agent with random default policy.

In this paper we explore different types of procedurally generated content to see if and how it can affect the gameplay experience of the player. We conduct a survey with different types of procedurally generated content in order to see if and how it affects the player when playing a game. The paper presents a simple turn based tactical strategy game where the objective of the player is to eliminate the enemy team. The player must achieve this goal in different circumstances: from a simple grid to a procedurally random generated levels. The experiments conducted will show the way procedurally generation will enhance the gameplay experience of the player and where it should be used.

The Wild West has been productized and remediated a number of times over the last hundred of yearsand even earlier than that. Computer and video games are a relative newcomer in this string of remediations, distilling the essential elements of the Western even further into archetypal scenes, characters and modes of play. Expanding on our previous works on genres, we study a colorful selection of Western-themed games, 31 in total, with the aim to reveal how a firmly established and codified theme affects gameplay. A well-established theme, such as the Western, manifests itself at very different depths based on the kind of game we are dealing withat minimum, it acts as an artificial surface that does not affect the gameplay at all. At the other end of the spectrum, the theme affects all the aspects of a game, ranging from its audiovisual presentation to the narrative and the actions available to the player.

Artificial intelligence allows computer systems to make decisions similar to those of humans. However, the expert knowledge that artificial intelligence systems have is rarely used to teach non-expert humans in a specific knowledge domain. In this paper, we want to explore this possibility by proposing a tool which presents and explains recommendations for playing board games generated by a Monte Carlo Tree Search algorithm combined with Neural Networks. The aim of the aforementioned tool is to showcase the information in an easily interpretable way and to effectively transfer knowledge: in this case, which movements should be avoided, and which action is recommended. Our system displays the state of the game in the form of a tree, showing all the movements available from the current state and a set of their successors. To convince and try to teach people, the tool offers a series of queries and all information available about every possible movement. In addition, it produces a brie...

This paper presents a method for generating complex problems that allow multiple non-obvious solutions for the Physical Travelling Salesman Problem (PTSP). PTSP is a single-player game adaptation of the classical Travelling Salesman Problem that makes use of a simple physics model: the player has to visit a number of waypoints as quickly as possible by navigating a ship in real time across an obstaclefilled two-dimensional map. The difficulty of this game depends on the distribution of waypoints and obstacles across the twodimensional plane. Due to the physics of the game, the shortest route is not necessarily the fastest, as the ship's momentum makes it difficult to turn sharply at speed. This paper proposes an evolutionary approach to obtaining maps where the optimal solution is not immediately obvious. In particular, any optimal route for these maps should differ distinctively from (a) the optimal distance-based TSP route and (b) the route that corresponds to always approaching the nearest waypoint first. To achieve this, the evolutionary algorithm CMA-ES is employed, where maps, indirectly represented as vectors of real numbers, are evolved to differentiate maximally between a game-playing agent that follows two or more different routes. The results presented in this paper show that CMA-ES is able to generate maps that fulfil the desired conditions.

This work studies the performance and the results of the application of Evolutionary Algorithms (EAs) for evolving the decision engine of a program, called in this context agent, which controls the player's behaviour in an real-time strategy game (RTS). This game was chosen for the Google Artificial Intelligence Challenge in 2011, and simulates battles between teams of ants in different types of maps or mazes. According to the championship rules the agents cannot save information from one game to the next, which makes impossible to implement an EA 'inside' the agent, i.e. on game time (or on-line), that is why in this paper we have evolved this engine off-line by means of an EA, used for tuning a set of constants, weights and probabilities which direct the rules. This evolved agent has fought against other successful bots which finished in higher positions in the competition final rank. The results show that, although the best agents are difficult to beat, our simple agent tuned with an EA can outperform agents which have finished 1000 positions above the untrained version.

Real-time strategy (RTS) games, such as Blizzard's StarCraft, are fast paced war simulation games in which players have to manage economies, control many dozens of units, and deal with uncertainty about opposing unit locations in real-time. Even in perfect information settings, constructing strong AI systems has been difficult due to enormous state and action spaces and the lack of good state evaluation functions and high-level action abstractions. To this day, good human players are still handily defeating the best RTS game AI systems, but this may change in the near future given the recent success of deep convolutional neural networks (CNNs) in computer Go, which demonstrated how networks can be used for evaluating complex game states accurately and to focus look-ahead search. In this paper we present a CNN for RTS game state evaluation that goes beyond commonly used material based evaluations by also taking spatial relations between units into account. We evaluate the CNN's performance by comparing it with various other evaluation functions by means of tournaments played by several state-of-the-art search algorithms. We find that, despite its much slower evaluation speed, the CNN based search performs significantly better compared to simpler but faster evaluations. These promising initial results together with recent advances in hierarchical search suggest that dominating human players in RTS games may not be far off.

The question of whether the correct algorithm is used for the problem at hand usually comes at the end of execution, when the algorithm's ability to solve the problem (or not) can be verified. But what if this question could be answered in advance, with enough notice to make changes in the approach in order for it to be more successful? This paper proposes a general agent performance prediction system, tested in real time within the context of the General Video Game AI framework. It is solely based on agent features, therefore removing potential human bias produced by game-based features observed in known games. Three different models can be queried while playing the game to determine whether the agent will win or lose, based on the current game state: early, mid and late game feature models. The models are trained on 80 games in the framework and tested on 20 new games, for 14 variations of 3 different methods. Results are positive, indicating that there is great scope for predicting the outcome of any given game.

This paper presents a new track of the General Video Game AI competition for generic Artificial Intelligence agents, which features both competitive and cooperative real time stochastic two player games. The aim of the competition is to directly test agents against each other in more complex and dynamic environments, where there is an extra uncertainty in a game, consisting of the behaviour of the other player. The framework, server functionality and general competition setup are analysed and the results of the experiments with several sample controllers are presented. The results indicate that currently Open Loop Monte Carlo Tree Search is the overall leading algorithm on this set of games.

General Video Game Playing is a sub-field of Game Artificial Intelligence, where the goal is to find algorithms capable of playing many different real-time games, some of them unknown a priori. In this scenario, the presence of domain knowledge must be severely limited, or the algorithm will overfit to the training games and perform poorly on the unknown games of the test set. Research in this area has been of special interest in the last years, with emerging contests like the General Video Game AI (GVG-AI) Competition. This paper introduces three different open loop techniques for dealing with this problem. First, a simple directed depth first search algorithm is employed as a baseline. Then, a tree search algorithm with a multi-armed bandit based tree policy is presented, followed by a Rolling Horizon Evolutionary Algorithm (RHEA) approach. In order to test these techniques, the games from the GVG-AI Competition framework are used as a benchmark, evaluation on a training set of 29 games, and submitting to the 10 unknown games at the competition website. Results show how the general game-independent heuristic proposed works well across all algorithms and games, and how the RHEA becomes the best evolutionary technique in the rankings of the test set.

Artificial intelligence allows computer systems to make decisions similar to those of humans. However, the expert knowledge that artificial intelligence systems have is rarely used to teach non-expert humans in a specific knowledge domain. In this paper, we want to explore this possibility by proposing a tool which presents and explains recommendations for playing board games generated by a Monte Carlo Tree Search algorithm combined with Neural Networks. The aim of the aforementioned tool is to showcase the information in an easily interpretable way and to effectively transfer knowledge: in this case, which movements should be avoided, and which action is recommended. Our system displays the state of the game in the form of a tree, showing all the movements available from the current state and a set of their successors. To convince and try to teach people, the tool offers a series of queries and all information available about every possible movement. In addition, it produces a brie...

Dynamic difficulty adjustment (DDA) is a method of automatically modifying a game’s features, behaviors, and scenarios in real-time, depending on the player’s skill, so that the player, when the game is very simple, does not feel bored or frustrated, when it is very difficult. The intent of the DDA is to keep the player engrossed till the end and to provide him/her with a challenging experience. In traditional games, difficulty levels increase linearly or stepwise during the course of the game. The features such as frequency, starting levels, or rates can be set only at the beginning of the game by choosing a level of difficulty. This can, however, result in a negative experience for players as they try to map a predecided learning curve. DDA attempts to solve this problem by presenting a customized solution for the gamers. This paper provides a review of the current approaches to DDA.

We describe a machine playtesting system that combines two paradigms of artificial intelligence-learning and tree search-and intends to place them in the hands of independent game developers. This integration approach has shown great success in Go-playing systems like AlphaGo and AlphaZero, but until now has not been available to those outside of artificial intelligence labs. Our system expands the Monster Carlo machine playtesting framework for Unity games by integrating its tree search capabilities with the behavior cloning features of Unity's Machine Learning Agents Toolkit. Because experience gained in one playthrough may now usefully transfer to other playthroughs via imitation learning, the new system overcomes a serious limitation of the older one with respect to stochastic games (when memorizing a single optimal solution is ineffective). Additionally, learning allows search-based automated play to be bootstrapped from examples of human play styles or even from the best of its own past experiences. In this paper we demonstrate that our framework discovers higher-scoring and more-representative play with minimal need for machine learning or search expertise.

Real-Time Strategy (RTS) games typically take place in a warlike setting and are accompanied with complicated game play. They are not only difficult for human players to master, but also provide a challenging platform for AI research. In a typical RTS game, such as StarCraft, WarCraft, or Age of Empires, knowing what the opponent is doing is a great advantage and sometimes an important key to win the game. For that, good scouting is required. As subsequent work for improving the scouting agent in our StarCraft AI bot-IceBot-the winner of the mixed division in Student StarCraft AI Tournament 2012, this paper proposes a method that applies potential flows to controlling scout units in StarCraft. The proposed method outperforms an existing scouting method as well as a modified version of this existing method and is comparable to scouting by human players.