Academia.eduAcademia.edu

Figure 5 – uploaded by Oleksandra Keehl

See full PDFdownloadDownload figure
Fig. 5. Top: Distribution of scores for play simulated without using search, just using a decision making policy based on our feature representation of game state. Bottom: Distribution of scores for play using search (making use of different rollout policies). The distribution of human expert scores (from 124 games) are included here. The two charts use the same horizontal axis scaling for game scores, but the vertical axes use different scales.

Figure 5 Top: Distribution of scores for play simulated without using search, just using a decision making policy based on our feature representation of game state. Bottom: Distribution of scores for play using search (making use of different rollout policies). The distribution of human expert scores (from 124 games) are included here. The two charts use the same horizontal axis scaling for game scores, but the vertical axes use different scales.

Related Figures (6)

Fig. 2. MC2 Process diagram. The two distinct procedures in MC2 are MCTS rollouts and behavior cloning. Each relies on the output of the other. The behavior cloning needs the behavior to clone (in the form of playtraces). These can either come from human examples or from the games played with MCTS. MCTS needs a way to make decisions during rollouts. This can be done either with a model produced by behavior cloning, by making choices at random or using a user-provided heuristic. Fig. 1. Software architecture: Green components are provided by the Monster Carlo framework, black components are provided by the Unity Machine Learning Agents Toolkit, and yellow components are project-specific details created by the game developer.
Fig. 2. MC2 Process diagram. The two distinct procedures in MC2 are MCTS rollouts and behavior cloning. Each relies on the output of the other. The behavior cloning needs the behavior to clone (in the form of playtraces). These can either come from human examples or from the games played with MCTS. MCTS needs a way to make decisions during rollouts. This can be done either with a model produced by behavior cloning, by making choices at random or using a user-provided heuristic. Fig. 1. Software architecture: Green components are provided by the Monster Carlo framework, black components are provided by the Unity Machine Learning Agents Toolkit, and yellow components are project-specific details created by the game developer.
Fig. 3. Screenshot of It’s Alive!, the Tetris-like game used in our experiments. During play, the player assembles monsters by rotating falling blocks. A monster comes alive when it has at least one head and one heart. Once alive it can optionally be collected to score points and free-up space. The player may choose to continue building up the monster to earn more points. The goal is to get the highest score after collecting five monsters. The game is lost if the pieces pile up to the top.
Fig. 3. Screenshot of It’s Alive!, the Tetris-like game used in our experiments. During play, the player assembles monsters by rotating falling blocks. A monster comes alive when it has at least one head and one heart. Once alive it can optionally be collected to score points and free-up space. The player may choose to continue building up the monster to earn more points. The goal is to get the highest score after collecting five monsters. The game is lost if the pieces pile up to the top.
Fig. 4. Game state representation of /t’s Alive!. Here we see what the state representation would look like for the pictured 2x3 field on the left. Seven binary channels with the dimensions matching those of the playfield represent the game state by binary encoding of each of the features we deemed relevant to make the decision on where to place the current block: whether a grid cell is occupied, whether the part in the cell has connections in one of four directions, and whether the part or the monster it belongs to has a head or a heart. The relevant information for the current and upcoming blocks is also captured (the green- and red-tinged numbers respectively in this illustration).
Fig. 4. Game state representation of /t’s Alive!. Here we see what the state representation would look like for the pictured 2x3 field on the left. Seven binary channels with the dimensions matching those of the playfield represent the game state by binary encoding of each of the features we deemed relevant to make the decision on where to place the current block: whether a grid cell is occupied, whether the part in the cell has connections in one of four directions, and whether the part or the monster it belongs to has a head or a heart. The relevant information for the current and upcoming blocks is also captured (the green- and red-tinged numbers respectively in this illustration).
Fig. 6. Summary statistics for each dataset of scores. With exception 0! MCTS with H1 and MCTS with H2, each pair of the results was statistically distinct with p < 0.009 according to a two-sided Mann-Whitney U test.
Fig. 6. Summary statistics for each dataset of scores. With exception 0! MCTS with H1 and MCTS with H2, each pair of the results was statistically distinct with p < 0.009 according to a two-sided Mann-Whitney U test.
Fig. 7. Game states resulting from four possible piece-landing actions. States a and b are much more desirable than c and d because nothing can be attached to the monster head in the latter two cases, making it permanently take up space in the field.
Fig. 7. Game states resulting from four possible piece-landing actions. States a and b are much more desirable than c and d because nothing can be attached to the monster head in the latter two cases, making it permanently take up space in the field.
Related topics:
Computer ScienceMachine Learning
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved