General Game Playing

We propose a method to guide a Monte Carlo search in the initial moves of the game of Go. Our method matches the current state of a Go board against clusters of board configurations that are derived from a large number of games played by experts. The main advantage of this method is that it does not require an exact match of the current board, and hence is effective for a longer sequence of moves compared to traditional opening books. We apply this method to two different open-source Go-playing programs. Our experiments show that this method, through its filtering or biasing the choice of a next move to a small subset of possible moves, improves play effectively in the initial moves of a game.

This document is based upon work from COST Action “A European Network to Leverage the Multi-Age Workforce” (LeverAge) CA22120, supported by COST (European Cooperation in Science and Technology). COST (European Cooperation in Science and Technology) is a funding agency for research and innovation networks. Our Actions help connect research initiatives across Europe and enable scientists to grow their ideas by sharing them with their peers. This boosts their research, career and innovation.

The multifaceted study of games spans diverse disciplines which have often operated in isolation. In
this era of rapid developments in computer science, games stand as testbeds for pioneering
methodologies, shaping and influencing techniques in economics, engineering, and AI. At the same
time, archaeologists, historians, and anthropologists explore the motivations underlying human play,
dissecting its social implications on both individual and societal levels. Furthermore, games have
played a central role in pedagogical development and are progressively acknowledged as integral
components of humanity’s cultural heritage.
The GameTable COST Action is an ongoing scientific network which started in October 2023, funded
by the European Cooperation in Science and Technology research scheme, aiming to gather scholars,
researchers and stakeholders from a variety of fields and career stages across academia, industry and
heritage institutions. GameTable aims to forge long-lasting international collaborations by
establishing an interdisciplinary network to develop new methodologies and applications for using
game AI to study, reconstruct, and preserve the intangible cultural heritage of games. 
The network proposes to approach AI techniques, archaeology, and gaming research into a cohesive
framework. In doing so, the group aspires to demonstrate how games are both form of entertainment
and repositories of cultural richness and historical narratives. Key challenges include innovative
approaches for studying and preserving heritage games, reconstructing missing rules in incomplete
games, simulating human-like play, applying AI to study historical games, developing novel methods
to identify unrecognized games measuring their evolution across space and time, developing
pedagogical tools, and leveraging AI in a culturally diverse way.

Improving the clarity of games allows players to spend more of their mental effort on strategic planning rather than the mundane bookkeeping of calculating legal moves. This article discusses techniques for achieving this, by making the design do the work rather than the player, and demonstrates this concept through example. Such techniques include visual design, simplifying rules, clarifying rules, harnessing emergent strategies, and minimising mental bookkeeping.

On 20 and 21 January 2025, the GameTable COST Action's Working Group 1 convened in London for a meeting focused on game-playing artificial intelligence (AI), including search algorithms, knowledge representation, and reinforcement learning. The first day featured participant presentations, while the second day was dedicated to plenary and small group discussions. A significant outcome was the identified need for accessible resources to bridge AI and cultural heritage research. This report provides a summary of the discussions and talks that took place during the meeting.

General Game Playing aims to create AI systems that can understand the rules of new games and learn to play them effectively without human intervention. The recent proposal for general game-playing robots extends this to AI systems that play games in the real world. Execution monitoring becomes a necessity when moving from a virtual to a physical environment, because in reality actions may not be executed properly and (human) opponents may make illegal game moves. We develop a formal framework for execution monitoring by which an action theory that provides an axiomatic description of a game is automatically embedded in a meta-game for a robotic player--called the arbiter -- whose role is to monitor and correct failed actions. This allows for the seamless encoding of recovery behaviours within a meta-game, enabling a robot to recover from these unexpected events.

Negotiations in which participants exchange offers based on their chosen positions can be extended to include dialogue about their interests. Revelation of negotiators' interests allows them to make more acceptable offers and perhaps propose possible alternative approaches toward each other's interests, both of which may result in mutually and individually beneficial outcomes. However, it can also expose their strategies, and possibly their dependencies on other negotiators toward the achievement of their goals. Revealing this information can leave them vulnerable to extortion or retribution, but it can also be used to gain sympathy or build a relationship of trust and reciprocity.

While current General Game Playing (GGP) systems facilitate useful research in Artificial Intelligence (AI) for game-playing, they are often somewhat specialized and computationally inefficient. In this paper, we describe an initial version of a "ludemic" general game system called Ludii, which has the potential to provide an efficient tool for AI researchers as well game designers, historians, educators and practitioners in related fields. Ludii defines games as structures of ludemes, i.e. highlevel, easily understandable game concepts. We establish the foundations of Ludii by outlining its main benefits: generality, extensibility, understandability and efficiency. Experimentally, Ludii outperforms one of the most efficient Game Description Language (GDL) reasoners, based on a propositional network, for all available games in the Tiltyard GGP repository.

While current General Game Playing (GGP) systems facilitate useful research in Artificial Intelligence (AI) for game-playing, they are often somewhat specialized and computationally inefficient. In this paper, we describe an initial version of a "ludemic" general game system called Ludii, which has the potential to provide an efficient tool for AI researchers as well game designers, historians, educators and practitioners in related fields. Ludii defines games as structures of ludemes, i.e. highlevel, easily understandable game concepts. We establish the foundations of Ludii by outlining its main benefits: generality, extensibility, understandability and efficiency. Experimentally, Ludii outperforms one of the most efficient Game Description Language (GDL) reasoners, based on a propositional network, for all available games in the Tiltyard GGP repository.

The goal of this dissertation is to propose a high level generic architecture for the development of agents able to effectively play games with strong social components and a mix of competition and cooperation. Traditional techniques used in the context of games include a combination of heuristics and searching strategies like minimax, Branch & Bound as well as Monte-Carlo approaches; however, these techniques are difficult to apply to this category of games, due to the often enormous search trees and the difficulty in calculating the value of a player's position or move. We propose a generic agent architecture that tackles the subjects of negotiation, trust and opponent modeling, simplifying the development of agents capable of playing these games effectively by introducing modules to handle these challenges in addition to a traditional strategic module. This architecture is split into four independent modules, taking inspiration from the structure of a wartime nation: the President, the Strategic Office, the Foreign Office and the Intelligence Office. We demonstrate the application of this architecture by instantiating it using two different games-Diplomacy and Werewolves of Miller's Hollow-and testing the obtained agents in a variety of scenarios against existing agents. The results obtained show that the architecture is generic enough to be applied in a wide variety of games. Furthermore, the inclusion of modules to handle negotiation, trust reasoning and opponent modeling allows for more effective agents. i I would like to thank my supervisor Henrique Lopes Cardoso, who was always ready to help me with useful advise and valuable criticism which allowed me to finish this dissertation. I would also like to thank the other teachers and professors that have accompanied me during the adventure of the last five years and who taught me many valuable lessons about computer science and software engineering and without which I wouldn't be where I am today. Dave de Jonge and André Ferreira were also extremely important for this work, as it is due to their work that a large part of this dissertation was possible, and for that they have my sincere thanks. Finally, I'd like to especially thank my friends and my family, who have always been there for me and supported me during some of the toughest moments of my life. João Marinheiro v vi "If you want to be incrementally better: Be competitive. If you want to be exponentially better: Be cooperative." Unknown Author vii viii Negotiation games with a mix of cooperation and competition are a specific set of games with interesting characteristics for AI research, especially in the areas of negotiation and trust. There is a large variety of negotiation games with cooperative elements, some examples being Diplomacy, Werewolves, Quo Vadis?, Santiago and Genoa. Important defining characteristics of these games are: • Large focus on forming coalitions and negotiating deals in order to improve one's chances of winning.

Attempts to develop generic approaches to game playing have been around for several years in the field of Artificial Intelligence. However, games that involve explicit cooperation among otherwise competitive players-cooperative negotiation games-have not been addressed by current approaches. Yet, such games provide a much richer set of features, related with social aspects of interactions, which make them appealing for envisioning real-world applications. This work proposes a generic agent architecture-Alpha-to tackle cooperative negotiation games, combining elements such as search strategies, negotiation, opponent modeling and trust management. The architecture is then validated in the context of two different games that fall in this category-Diplomacy and Werewolves. Alpha agents are tested in several scenarios, against other state-of-the-art agents. Besides highlighting the promising performance of the agents, the role of each architectural component in each game is assessed.

The goal of this dissertation is to propose a high level generic architecture for the development of agents able to effectively play games with strong social components and a mix of competition and cooperation. Traditional techniques used in the context of games include a combination of heuristics and searching strategies like minimax, Branch & Bound as well as Monte-Carlo approaches; however, these techniques are difficult to apply to this category of games, due to the often enormous search trees and the difficulty in calculating the value of a player's position or move. We propose a generic agent architecture that tackles the subjects of negotiation, trust and opponent modeling, simplifying the development of agents capable of playing these games effectively by introducing modules to handle these challenges in addition to a traditional strategic module. This architecture is split into four independent modules, taking inspiration from the structure of a wartime nation: the President, the Strategic Office, the Foreign Office and the Intelligence Office. We demonstrate the application of this architecture by instantiating it using two different games-Diplomacy and Werewolves of Miller's Hollow-and testing the obtained agents in a variety of scenarios against existing agents. The results obtained show that the architecture is generic enough to be applied in a wide variety of games. Furthermore, the inclusion of modules to handle negotiation, trust reasoning and opponent modeling allows for more effective agents. i I would like to thank my supervisor Henrique Lopes Cardoso, who was always ready to help me with useful advise and valuable criticism which allowed me to finish this dissertation. I would also like to thank the other teachers and professors that have accompanied me during the adventure of the last five years and who taught me many valuable lessons about computer science and software engineering and without which I wouldn't be where I am today. Dave de Jonge and André Ferreira were also extremely important for this work, as it is due to their work that a large part of this dissertation was possible, and for that they have my sincere thanks. Finally, I'd like to especially thank my friends and my family, who have always been there for me and supported me during some of the toughest moments of my life. João Marinheiro v vi "If you want to be incrementally better: Be competitive. If you want to be exponentially better: Be cooperative." Unknown Author vii viii Negotiation games with a mix of cooperation and competition are a specific set of games with interesting characteristics for AI research, especially in the areas of negotiation and trust. There is a large variety of negotiation games with cooperative elements, some examples being Diplomacy, Werewolves, Quo Vadis?, Santiago and Genoa. Important defining characteristics of these games are: • Large focus on forming coalitions and negotiating deals in order to improve one's chances of winning.

We present a new technique to compress pattern databases to provide consistent heuristics without loss of information. We store the heuristic estimate modulo three, requiring only two bits per entry or in a more compact representation, only 1.6 bits. This allows us to store a pattern database with more entries in the same amount of memory as an uncompressed pattern database. These compression techniques are most useful where lossy compression using cliques or their generalization is not possible or where adjacent entries in the pattern database are not highly correlated. We compare both techniques to the best existing compression methods for the Top-Spin puzzle, Rubik's cube, the 4-peg Towers of Hanoi problem, and the 24 puzzle. Under certain conditions, our best implementations for the Top-Spin puzzle and Rubik's cube outperform the respective state of the art solvers by a factor of four.

This short paper describes an ongoing research project that requires the automated self-play learning and evaluation of a large number of board games in digital form. We describe the approach we are taking to determine relevant features, for biasing MCTS playouts for arbitrary games played on arbitrary geometries. Benefits of our approach include efficient implementation, the potential to transfer learnt knowledge to new contexts, and the potential to explain strategic knowledge embedded in features in human-comprehensible terms.

This short paper describes an ongoing research project that requires the automated self-play learning and evaluation of a large number of board games in digital form.We describe the approach we are taking to determine relevant features, for biasing MCTS playouts for arbitrary games played on arbitrary geometries. Benefits of our approach include efficient implementation, the potential to transfer learnt knowledge to new contexts, and the potential to explain strategic knowledge embedded in features in human-comprehensible terms.

Humans tend to learn complex abstract concepts faster if examples are presented in a structured manner. For instance, when learning how to play a board game, usually one of the first concepts learned is how the game ends, i.e. the actions that lead to a terminal state (win, lose or draw). The advantage of learning end-games first is that once the actions which lead to a terminal state are understood, it becomes possible to incrementally learn the consequences of actions that are further away from a terminal state-we call this an end-game-first curriculum. Currently the state-of-the-art machine learning player for general board games, AlphaZero by Google DeepMind, does not employ a structured training curriculum; instead learning from the entire game at all times. By employing an end-game-first training curriculum to train an AlphaZero inspired player, we empirically show that the rate of learning of an artificial player can be improved when compared to a player not using a training curriculum.

This paper introduces a new negotiating agent model for automated negotiation. We focus on applications without time pressure with multidimensional negotiation on both continuous and discrete domains. The agent bidding strategy relies on Monte Carlo Tree Search, which is a trendy method since it has been used with success on games with high branching factor such as Go. It also exploits opponent modeling techniques thanks to Gaussian process regression and Bayesian learning. Evaluation is done by confronting the existing agents that are able to negotiate in such context: Random Walker, Tit-for-tat and Nice Titfor-Tat. None of those agents succeeds in beating our agent. Also, the modular and adaptive nature of our approach is a huge advantage when it comes to optimize it in specific applicative contexts.

This paper introduces a new Negotiating Agent for automated negotiation on continuous domains and without considering a specified deadline. The agent bidding strategy relies on Monte Carlo Tree Search, which is a trendy method since it has been used with success on games with high branching factor such as Go. It uses two opponent modeling techniques for its bidding strategy and its utility: Gaussian process regression and Bayesian learning. Evaluation is done by confronting the existing agents that are able to negotiate in such context: Random Walker, Tit-fortat and Nice Tit-for-Tat. None of those agents succeeds in beating our agent; moreover the modular and adaptive nature of our approach is a huge advantage when it comes to optimize it in specific applicative contexts.

This paper describes a co-evolutionary algorithm for generating simple spatially oriented tactics and considers whether students can learn better by playing against co-evolved opponents or by playing against an expert system or other similar hard-coded opponent. Although a number of artificially intelligent tutoring and e-learning systems exist, our work looks at using co-evolution to generate competent opponents for human students to learn from. This paper describes and discusses early results on using computationally intelligent opponents for tactical training of human students. Initial results indicate that the learning environment for human players, measured by game difficulty and transfer of training, is comparable across coevolved and hard-coded computer opponents.

Real-Time Strategy (RTS) games have become an attractive domain for AI research in recent years, due to their dynamic, multi-agent and multi-objective environments. Micromanagement, a core component of many RTS games, involves the control of multiple agents to accomplish goals that require fast, real time assessment and reaction. In this paper, we present the application and evaluation of a Neuroevolution technique in evolving micromanagement agents for the RTS game Starcraft: Brood War (SC:BW). The NeuroEvolution of Augmented Topologies (NEAT) algorithm, both in its standard form and its real-time variant (rtNEAT) is comparatively evaluated in micromanagement tasks. Preliminary results suggest the general viability of these techniques in comparison to traditional, non-adaptive AI. Further analysis of each algorithm identified differences in task performance and learning rate.

General Game Playing aims at AI systems that can understand the rules of new games and learn to play them effectively without human intervention. Our paper takes the first step towards general game-playing robots, which extend this capability to AI systems that play games in the real world. We develop a formal model for general games in physical environments and provide a systems architecture that allows the embedding of existing general game players as the "brain" and suitable robotic systems as the "body" of a general game-playing robot. We also report on an initial robot prototype that can understand the rules of arbitrary games and learns to play them in a fixed physical game environment.

We present an approach to procedurally generate the narrative of a simple murder mystery. As a basis for the simulation, we use a rule evaluation system inspired by Ceptre, which employs linear logic to resolve valid actions during each step of the simulation. We extend Ceptre’s system with a concept of believable agents to make consecutive actions appear to have a causal connection so that players can comprehend the flow of events. The parts of the generated narratives are then presented to a player whose task it is to figure out who the murderer in this story could have been. Rather than aiming to replace highly authored narratives, this project generates puzzles, which may contain emerging arcs of a story as perceived by the player. While we found that even a simple rule set can create stories that are interesting to reason about, we expect that this type of system is flexible enough to create considerably more engaging stories if enough time is invested in authoring more complex...

We present a new general board game (GBG) playing and learning framework. GBG defines the common interfaces for board games, game states and their AI agents. It allows one to run competitions of different agents on different games. It standardizes those parts of board game playing and learning that otherwise would be tedious and repetitive parts in coding. GBG is suitable for arbitrary 1-, 2-,. .. , N-player board games. It makes a generic TD(λ)-n-tuple agent for the first time available to arbitrary games. On various games, TD(λ)-n-tuple is found to be superior to other generic agents like MCTS. GBG aims at the educational perspective, where it helps students to start faster in the area of game learning. GBG aims as well at the research perspective by collecting a growing set of games and AI agents to assess their strengths and generalization capabilities in meaningful competitions. Initial successful educational and research results are reported.

The success of human civilization is rooted in our ability to cooperate by communicating and making joint plans. We study how artificial agents may use communication to better cooperate in Diplomacy, a long-standing AI challenge. We propose negotiation algorithms allowing agents to agree on contracts regarding joint plans, and show they outperform agents lacking this ability. For humans, misleading others about our intentions forms a barrier to cooperation. Diplomacy requires reasoning about our opponents' future plans, enabling us to study broken commitments between agents and the conditions for honest cooperation. We find that artificial agents face a similar problem as humans: communities of communicating agents are susceptible to peers who deviate from agreements. To defend against this, we show that the inclination to sanction peers who break contracts dramatically reduces the advantage of such deviators. Hence, sanctioning helps foster mostly truthful communication, despite conditions that initially favor deviations from agreements. Coordination, cooperation and negotiation play a key role in our everyday lives, from small-scale problems such as safely driving on roads and scheduling meetings to large-scale efforts such as international trade or mediating peace. A key driver to the success of humans as a species is our ability to cooperate with others 1,2. Systems based on artificial intelligence (AI) control a growing part of our lives, from personal assistants to high-stake decisions such as authorizing loans or automated job market screening 3,4. Cooperation and negotiation are central to AI 5-11 , and AI systems already affect human trade and negotiation through algorithmic trading and bidding 10,12-16. It is thus imperative that we endow our AI systems with the tools to coordinate and negotiate with others 3,7,11,17,18. Game playing has been a focus area of AI since its inception. Progress on search, reinforcement learning, and game theory 9,19-21 led to successes in Chess 22 , Go 23 , Poker 24 , control 25 , and video games 26,27. However, the majority of such work deals with two-player games that are fully competitive (zero-sum), which are mathematically easier to analyze 18,28 , but cannot capture alliance formation and negotiation. Similarly, work on games where agent goals are fully aligned 29,30 , or two teams of agents engaged in a competition 27,31,32 , lacks the need to negotiate. In contrast, many real-life domains require negotiation as the goals of participants only partially align. These domains exhibit tensions between cooperation and competition 7,33-35 , making them harder to tackle using AI agents 21,36,37. Communication has a key role in such settings as it enables us to share beliefs, goals and intents with others, allowing us to negotiate, form alliances and find mutually beneficial agreements 10. Diplomacy 38 is a prominent 7-player board game that focuses on communication, cooperation and negotiation. An introduction to the rules of Diplomacy is given in Supplementary Note 1. Diplomacy is played on a map of Europe partitioned into provinces, some of which are special and marked as Supply Centers. Each player attempts to own the majority of the supply centers, and controls multiple units (armies or fleets). A unit may support another unit (owned by the same or another player), allowing it to overcome resistance by other units. Due to the inter-dependencies between units, players stand to gain by

This work compares the learning of linear evaluation functions using preference learning versus least squares temporal difference learning, LSTD(λ), from samples of game trajectories. The game trajectories are taken from human competitions held by the French Othello Federation 1. The raw board positions are used to create a linear evaluation function to illustrate the key difference between the two learning approaches. The results show that the policies learned, using exactly the same game trajectories, can be quite different. For the simple set of features used, preference learning produces policies that better capture the behaviour of expert players, and also lead to higher levels of play when compared to LSTD(λ).

This paper highlights an experiment to see how standard Monte Carlo Tree Search handles simple cooperative problems with no prior or provided knowledge. These problems are formed from a simple grid world that has a set of goals, doors and buttons as well as walls that cannot be walked through. Two agents have to reach every goal present on the map. For a door to be open, an agent must be present on at least one of the buttons that is linked to it. When laid out correctly, the world requires each agent to do certain things at certain times in order to achieve the goal. With no modification to allow communication between the two agents, Monte Carlo Tress Search performs well and very "purposefully" when given enough computational time.

Recent successes of neural networks in solving combinatorial problems and games like Go, Poker and others inspire further attempts to use deep learning approaches in discrete domains. In the field of automated planning, the most popular approach is informed forward search driven by a heuristic function which estimates the quality of encountered states. Designing a powerful and easily-computable heuristics however is still a challenging problem on many domains. In this paper, we use machine learning to construct such heuristic automatically. We train a neural network to predict a minimal number of moves required to solve a given instance of Rubik's cube. We then use the trained network as a heuristic distance estimator with a standard forward-search algorithm and compare the results with other heuristics. Our experiments show that the learning approach is competitive with state-of-the-art and might be the best choice in some use-case scenarios.

Monte Carlo Tree Search methods provide a general framework for modeling decision problems by randomly sampling the decision space and constructing a search tree according to the sampling results. Artificial Intelligences employing these methods in games with massive decision spaces such as Go and Settlers of Cataan have recently demonstrated far superior results compared to the previous classic game theory approaches [Browne et al. 1]. We apply Monte Carlo Tree Search methods, particularly the UCT variant, to an online version of the popular board game Risk.

Mediation is a process in which two parties agree to resolve their dispute by negotiating over alternative solutions presented by a mediator. In order to construct such solutions, the mediator brings more information and knowledge, and, if possible, resources to the negotiation table. In order to do so, the mediator faces the challenge of determining which information is relevant to the current problem, given a vast database of knowledge. The contribution of this paper is the automated mediation machinery to resolve this issue. We define the concept of a Mediation Problem and show how it can be described in Game Description Language (GDL). Furthermore, we present an algorithm that allows the mediator to efficiently determine which information is relevant to the problem and collect this information from the negotiating agents. We show with several experiments that this algorithm is much more efficient than the naive solution that simply takes all available knowledge into account.

: Games provide fertile research domains for algorithmic research. Often, game research helps solve real-world problems through the testing and refinement of search algorithms in game domains. Other times, game research finds limits for certain algorithms. For example, the game of Go proved intractable for the Min-Max with Alpha-Beta pruning algorithm leading to the popularity of Monte-Carlo based search algorithms. Although effective in Go, and game domains once ruled by Alpha-Beta such as Lines of Action, Monte-Carlo methods appear to have limits too as they fall short in tactical domains such as Hex and Chess. In a continuation of this type of research, two new games, Crossings and Epaminondas, are presented, analyzed and used to test two Monte-Carlo based algorithms: Upper Confidence Bounds applied to Trees (UCT) and Heuristic Guided UCT (HUCT). Results indicate that heuristic knowledge can positively affect UCT?s performance in the lower complexity domain of Crossings. However,...

Reinforcement Learning (RL) gained a huge amount of popularity in computer science; applied in fields such as gaming, intelligent robots, remote sensing, and so on. The objective of reinforcement learning is to generate the optimal policy. The main problem of that optimal policy is that it is not fully guaranteed to be satisfied all the system specifications. Model checking is a technique to verify the system to meet the system specifications. PRISM-games is one of the model-checking tools that is used to verify the probabilistic system with competitive or collaborative behavior. Safe Reinforcement Learning via Shielding is a method that uses shield to restrict the action of the RL agent if it violates the specification using temporal logic. This paper presents to compare the winning strategies between three agents; Monte-Carlo Tree Search agent (MCTS), RL agent and shielded RL agent (SRL) which uses PRISM-games to restrict the action based on Tic-Tac-Toe game. Over thousand times of simulations has been made, the experiments show that MCTS agent has the highest win rate compared to other agents, but the losing rate of the shielded agent is reduced by using PRISM-games.

In this work, we propose a Dynamic Difficulty Adjustment methodology to achieve automatic video game balance. The balance task is modeled as a meta game, a game where actions change the rules of another base game. Based on the model of Reinforcement Learning (RL), an agent assumes the role of a game master and learns its optimal policy by playing the meta game. In this new methodology we extend traditional RL by adding the existence of a meta environment whose state transition depends on the evolution of a base environment. In addition, we propose a Multi Agent System training model for the game master agent, where it plays against multiple agent opponents, each with a distinct behavior and proficiency level while playing the base game. Our experiment is conducted on an adaptive grid-world environment in singleplayer and multiplayer scenarios. Our results are expressed in twofold: (i) the resulting decision making by the game master through gameplay, which must comply in accordance to an established balance objective by the game designer; (ii) the initial conception of a framework for automatic game balance, where the balance task design is reduced to the modulation of a reward function (balance reward), an action space (balance strategies) and the definition of a balance space state.

Although utilising computers to play board games has been a topic of research for many decades, the recent rapid developments in the field of reinforcement learning-like AlphaZero (and variants)-brought unprecedented progress in games such as chess and Go. However, the efficiency of this process remains unknown. In this work, we analyse the cost and efficiency of the AlphaZero approach when building a new game engine. Thus, we present our experience building AZ-Hive, an AlphaZero-based playing engine for the game of Hive. Using only the rules of the game and a quality of play assessment, AZ-Hive learns to play the game from scratch. Getting AZ-Hive up and running requires encoding the game in AlphaZero, i.e., capturing the board, the game state, the rules and the assessment of play-quality. And different encodings lead to significantly different AZ-Hive engines, with very different performance results. Thus, we propose a design space for configuring AZ-Hive, and demonstrate the costs and benefits of different configurations in this space. We find that different configurations lead to a less or more competitive playing-engine, but the training and evaluation for different such engines is prohibitively expensive. Moreover, no systematic, efficient exploration or pruning of the space is possible. In turn, an exhaustive exploration can easily take tens of training-years. CCS Concepts • Computer systems organization → Multicore architectures; • Software and its engineering → Software performance; • Computing methodologies → Parallel programming languages; Massively parallel algorithms; Artificial intelligence.

Game playing programming assignments can provide useful hands-on learning experiences for teaching search tree programming techniques, space efficient data representation, and heuristic evaluation functions. However, a number of issues arise with game playing assignments that are not relevant to the focus of learning the AI material. Among these are interface development and agent communication. This paper describes the implementation of a network game playing system that allows students to center their attention on the relevant AI topics. Student evaluations and test scores over the last three years indicate that the use of this game playing system has enhanced the learning of AI concepts.

In this work, we show the first results of a project where a combinatorial mobile application is used as a tool to gather users’ data, allowing some understanding about the learning behaviour of users solving combinatorial tasks, in particular, related with the NIM game. To obtain the results, 6.514 games and 29.667 moves where analysed and players where considered as production units, that transform input resources, i.e. attempts, moves of game pieces, number of games played; into output results, e.g. number of victories. This approach may give some insights to the use of digital games as research tools in Education.

Recently, the seminal algorithms AlphaGo and Al-phaZero have started a new era in game learning and deep reinforcement learning. While the achievements of AlphaGo and AlphaZero-playing Go and other complex games at super human level-are truly impressive, these architectures have the drawback that they require high computational resources. Many researchers are looking for methods that are similar to AlphaZero, but have lower computational demands and are thus more easily reproducible. In this paper, we pick an important element of AlphaZerothe Monte Carlo Tree Search (MCTS) planning stage-and combine it with temporal difference (TD) learning agents. We wrap MCTS for the first time around TD n-tuple networks and we use this wrapping only at test time to create versatile agents that keep at the same time the computational demands low. We apply this new architecture to several complex games (Othello, ConnectFour, Rubik's Cube) and show the advantages achieved with this AlphaZero-inspired MCTS wrapper. In particular, we present results that this agent is the first one trained on standard hardware (no GPU or TPU) to beat the very strong Othello program Edax up to and including level 7 (where most other learning-from-scratch algorithms could only defeat Edax up to level 2).

Some complex problems can be modeled using more than one type of device thus having some interaction between them to represent their behavior. From this perspective, we do not have a common formulation to represent both the formalism and its interaction. The purpose of this paper is to fill in this gap by proposing a formulation that represents, for a group of devices, their behavior and interactions. We call this formalism as "Cooperating Adaptive Devices". For illustration purposes, we presented an application where its behavior can be modeled using this formalism.

A formal device is said to be adaptive whenever its behavior changes dynamically, in a direct response to its input stimuli, without interference of external agents, even its users. In order to achieve this feature, adaptive devices have to be self-modifiable. In other words, any possible changes in the device's behavior must be known at their full extent at any

This paper highlights an experiment to see how standard Monte Carlo Tree Search handles simple cooperative problems with no prior or provided knowledge. These problems are formed from a simple grid world that has a set of goals, doors and buttons as well as walls that cannot be walked through. Two agents have to reach every goal present on the map. For a door to be open, an agent must be present on at least one of the buttons that is linked to it. When laid out correctly, the world requires each agent to do certain things at certain times in order to achieve the goal. With no modification to allow communication between the two agents, Monte Carlo Tress Search performs well and very "purposefully" when given enough computational time.

AI techniques are already widely used in game software to provide computer-controlled opponents for human players. However, game design is a more-challenging problem than game play. Designers typically expend great effort to ensure that their games are balanced and challenging. Dynamic game-balancing techniques have been developed to modify a game-engine’s parameters in response to user play. In this paper we describe a first attempt at using AI techniques to design balanced board games like checkers and Go by modifying the rules of the game, not just the rule parameters. Our approach involves the use of a commercial general game-playing (GGP) engine that plays according to rules that are specified in a general game-definition language. We use a genetic algorithm (GA) to search the space of game rules, looking for turn-based board games that are well balanced, i.e., those that the GGP engine in self-play finds equally hard to win from either side and rarely draws. The GA finds bette...

We present initial research towards procedural generation of Simplified Boardgames and translating them into an efficient GDL code. This is a step towards establishing Simplified Boardgames as a comparison class for General Game Playing agents. To generate playable, human readable, and balanced chess-like games we use an adaptive evolutionary algorithm with the fitness function based on simulated playouts. In future, we plan to use the proposed method to diversify and extend the set of GGP tournament games by those with fully automatically generated rules.

We formalize Simplified Boardgames language, which describes a subclass of arbitrary board games. The language structure is based on the regular expressions, which makes the rules easily machine-processable while keeping the rules concise and fairly human-readable.

In many games, moves consist of several decisions made by the player. These decisions can be viewed as separate moves, which is already a common practice in multi-action games for efficiency reasons. Such division of a player move into a sequence of simpler / lower level moves is called splitting. So far, split moves have been applied only in forementioned straightforward cases, and furthermore, there was almost no study revealing its impact on agents' playing strength. Taking the knowledge-free perspective, we aim to answer how to effectively use split moves within Monte-Carlo Tree Search (MCTS) and what is the practical impact of split design on agents' strength. This paper proposes a generalization of MCTS that works with arbitrarily split moves. We design several variations of the algorithm and try to measure the impact of split moves separately on efficiency, quality of MCTS, simulations, and action-based heuristics. The tests are carried out on a set of board games and...

In this work, we presented a study regarding two important aspects of evolving feature-based game evaluation functions: the choice of genome representation and the choice of opponent used to test the model. We compared three representations. One simpler and more limited, based on a vector of weights that are used in a linear combination of predefined game features. And two more complex, based on binary and n-ary trees. On top of this test, we also investigated the influence of fitness defined as a simulation-based function that: plays against a fixed weak opponent, plays against a fixed strong opponent, and plays against the best individual from the previous population. For a testbed, we have chosen a recently popular domain of digital collectible card games. We encoded our experiments in a programming game, Legends of Code and Magic, used in Strategy Card Game AI Competition. However, as the problems stated are of general nature we are convinced that our observations are applicable in the other domains as well.

We present the technical side of reasoning in Regular Boardgames (RBG) language-a universal General Game Playing (GGP) formalism for the class of finite deterministic games with perfect information, encoding rules in the form of regular expressions. RBG serves as a research tool that aims to aid in the development of generalized algorithms for knowledge inference, analysis, generation, learning, and playing games. In all these tasks, both generality and efficiency are important. In the first part, this paper describes optimizations used by the RBG compiler. The impact of these optimizations ranges from 1.7 to even 33-fold efficiency improvement when measuring the number of possible game playouts per second. Then, we perform an in-depth efficiency comparison with three other modern GGP systems (GDL, Ludii, Ai Ai). We also include our own highly optimized game-specific reasoners to provide a point of reference of the maximum speed. Our experiments show that RBG is currently the fastest among the abstract general game playing languages, and its efficiency can be competitive to common interface-based systems that rely on handcrafted game-specific implementations. Finally, we discuss some issues and methodology of computing benchmarks like this.

The speed of game rules processing plays an essential role in the performance of a General Game Playing (GGP) agent. Propositional Networks (propnets) are an example of a highly efficient representation of game rules. So far, in GGP, only software implementations of propnets have been proposed and investigated. In this paper, we present the first implementation of propnets on Field-Programmable Gate Arrays (FP-GAs), showing that they perform between 25 and 58 times faster than a software-propnet for most of the tested games. We also integrate the FPGA-propnet within an MCTS agent, discussing the challenges of the process, and possible solutions for the identified shortcomings.