RoboCup 2D

MarliK is a 2D soccer simulation team which has been participating in RoboCup competitions since 2008. In this paper we briefly describe our latest developments which are mainly focused on optimizing our algorithms and making our dribble skills more dynamic. Also although still in the preliminary stages, we are looking forward to develop some proper level of learning in our team which will be used to improve the recently optimized dribble algorithm and also our existing block algorithm in next stages in order to improve the goal of the more realistic simulated soccer agents.

O objetivo deste trabalho foi aplicar e analisar os efeitos do aprendizado por reforço na otimização de tomadas decisões de um sistema multiagente cooperativo.É apresentada uma metodologia de modelagem da técnica de aprendizado por reforço para times de futebol de robôs 2D. A implementação da estratégia de aprendizagem consistiu de quatro etapas: definição das ações dos agentes; definição dos estados do ambiente no qual os agentes estão inseridos; definição dos valores dos reforços; implementação no simulador RcSoccerSim da Robocup de futebol de robôs. Os testes estatísticos foram utilizados para verificar o comportamento do time de robôs durante todo o processo de aprendizado. A análise se deu verificando a evolução de desempenho do sistema multiagente como um todo, através de estudos do saldo de gols alcançado em cada jogo. Além disso, a performance individual de cada agente também foi quantificada. Através dos testes de análise de variância e comparações múltiplas foi possível quantificar quais agentes sofreram alterações de performance ao longo do processo de otimização.

 This document describes a simulated robotic soccer team (GPR-2D). The team is implemented using an Artificial Intelligence (AI) strategy to modify the behavior of players of one team of soccer robots of the RoboCup 2D simulation category. This strategy used is a Reinforcement Learning (RL) algorithm, Q-learning. This algorithm define how each agent should act according to the characteristics of the opponent players, selecting the best action to take in a given state of the environment.

This document presents a novel approach to control the attack behavior of a team of simulated soccer playing robot of the Robocup 2D category. The presented approach modifies the behavior of each player only when in the state "controlling the ball". The approach is based on a modified Q-Learning algorithm that implements a continuous machine learning process. After an initial learning phase, each player uses its previous experience during the simulation of the game to decide if it should dribble, pass or kick the ball to the adversary goal. Simulation results show that the proposed approach is capable of learn how to score goals, even when faced with the-champion of the previous world championship.

This work presents a new approach that allows the use of cases in a case base as heuristics to speed up Multiagent Reinforcement Learning algorithms, combining Case Based Reasoning (CBR) and Multiagent Reinforcement Learning (MRL) techniques. This approach, called Case Based Heuristically Accelerated Multiagent Reinforcement Learning (CB-HAMRL), builds upon an emerging technique, Heuristic Accelerated Reinforcement Learning (HARL), in which RL methods are accelerated by making use of heuristic information. CB-HAMRL is a subset of MRL that makes use of a heuristic function H derived from a case base, in a Case Based Reasoning manner. An algorithm that incorporates CBR techniques into the Heuristically Accelerated Minimax-Q is also proposed and a set of empirical evaluations were conducted in a simulator for the robot soccer domain, comparing the three solutions for this problem: MRL, HAMRL and CB-HAMRL. Experimental results show that using CB-HAMRL, the agents learn faster than using RL or HAMRL methods.

 This document describes a simulated robotic soccer team (GPR-2D). The team is implemented using an Artificial Intelligence (AI) strategy to modify the behavior of players of one team of soccer robots of the RoboCup 2D simulation category. This strategy used is a Reinforcement Learning (RL) algorithm, Q-learning. This algorithm define how each agent should act according to the characteristics of the opponent players, selecting the best action to take in a given state of the environment.

As RoboCupJunior reached a decade mark in 2011, we feel the need for examining the current situation after 12 revisions and modifications to the league rules and structures since its launch in 2000. RoboCupJunior International is now attracting over 250 teams involving approximately 1,000 students originating from more than 30 countries. This paper aims to report on the progress achieved thus far, both technologically and educationally, and the issues currently addressed, together with suggestions for the future of RoboCupJunior.

Setplays are predefined collaborative coordinate actions that players from any sport can use to gain advantage over its adversaries. Recently, a complete framework for creation and execution of this kind of coordinate behavior by teams composed of multiple independent agents was launched as free software (the Setplay Framework). In this paper, an approach based on Reinforcement Learning(RL) is proposed, that allows the use of experience to devise the better course of action in setplays with multiple choices. Simulations results show that the proposed approach allows a team of simulated agents to improve its performance against a known adversary team, achieving better results than previously proposed approaches using RL.

Este artigo descreve uma pesquisa sobre controladores fuzzy para o posicionamento do goleiro sem bola no futebol de robôs simulado em duas dimensões. Tal pesquisa foi realizada com o intuito de melhorar o desempenho do goleiro, mais especificamente, aumentar a quantidade de bolas agarradas pelo mesmo, em detrimento aos gols sofridos, melhorando seu posicionamento para recepcionar a bola. Para validar a pesquisa, foram simuladas 30 partidas contra alguns dos melhores times do mundo que participam da RoboCup. Os resultados obtidos são apresentados e as conclusões tomadas são discutidas, bem como são oferecidas sugestões de trabalhos futuros.

Multi-agent robotic competitions such as RoboCup provide the motivation for a developmental research agenda–one that focuses on the evolution of complete working systems and their cognitive architectures. In this paper, we describe the components and integration of one such system–the 2010 RoboCup Standard Platform League entry rUNSWift. The realtime control architecture employed consists of a hierarchy of modules that implement functions of perception, world-modelling and behaviour generation. The ...

This document presents a novel approach to control the attack behavior of a team of simulated soccer playing robot of the Robocup 2D category. The presented approach modifies the behavior of each player only when in the state "controlling the ball". The approach is based on a modified Q-Learning algorithm that implements a continuous machine learning process. After an initial learning phase, each player uses its previous experience during the simulation of the game to decide if it should dribble, pass or kick the ball to the adversary goal. Simulation results show that the proposed approach is capable of learn how to score goals, even when faced with the-champion of the previous world championship.

A new trend in automation is to deploy so-called cyberphysical systems (CPS) which combine computation with physical processes. The novel RoboCup Logistics League Sponsored by Festo (LLSF) aims at a such CPS logistic scenarios in an automation setting. A team of robots has to produce products from a number of semi-finished products which they have to machine during the game. Different production plans are possible and the robots need to recycle scrap byproducts. This way, the LLSF is a very interesting league offering a number of challenging research questions for planning, coordination, or communication in an application-driven scenario. In this paper, we outline the objectives of the LLSF and present steps for developing the league further towards a benchmark for logistics scenarios for CPS. As a major milestone we present the new automated referee system which helps in governing the game play as well as keeping track of the scored points in a very complex factory scenario.

A new trend in automation is to deploy so-called cyberphysical systems (CPS) which combine computation with physical processes. The novel RoboCup Logistics League Sponsored by Festo (LLSF) aims at a such CPS logistic scenarios in an automation setting. A team of robots has to produce products from a number of semi-finished products which they have to machine during the game. Different production plans are possible and the robots need to recycle scrap byproducts. This way, the LLSF is a very interesting league offering a number of challenging research questions for planning, coordination, or communication in an application-driven scenario. In this paper, we outline the objectives of the LLSF and present steps for developing the league further towards a benchmark for logistics scenarios for CPS. As a major milestone we present the new automated referee system which helps in governing the game play as well as keeping track of the scored points in a very complex factory scenario.

The Logistics League Sponsored by Festo (LLSF) is a RoboCup league focusing on in-factory logistics applications involving task-level planning, scheduling, and automation in an industrial production workflow scenario. It is meant to spawn interest in industry for current robotics research and to provide a benchmarking domain for such applications. In this paper, we first describe the current set of rules used for the competition in 2013, outlining several advancements like variant production and (almost) autonomous judging through a referee box. We describe and propose changes for 2014, including doubling the field-size, multiple teams competing on the field at the same time, and the introduction of physical processing machinery replacing the signal light metaphor used so far, addressing the recommendations by RoboCup community members and casual visitors alike.

A new trend in automation is to deploy so-called cyberphysical systems (CPS) which combine computation with physical processes. The novel RoboCup Logistics League Sponsored by Festo (LLSF) aims at a such CPS logistic scenarios in an automation setting. A team of robots has to produce products from a number of semi-finished products which they have to machine during the game. Different production plans are possible and the robots need to recycle scrap byproducts. This way, the LLSF is a very interesting league offering a number of challenging research questions for planning, coordination, or communication in an application-driven scenario. In this paper, we outline the objectives of the LLSF and present steps for developing the league further towards a benchmark for logistics scenarios for CPS. As a major milestone we present the new automated referee system which helps in governing the game play as well as keeping track of the scored points in a very complex factory scenario.

RoboCup was created in 1996 by a group of Japanese, American, and European Artificial Intelligence and Robotics researchers with a formidable, visionary longterm
challenge: “By 2050 a team of robot soccer players will beat the human World Cup champion team.” At that time, in
the mid 90s, when there were very few effective mobile robots and the Honda P2 humanoid robot was presented to a stunning public for the first time also in 1996, the RoboCup challenge,
set as an adversarial game between teams of autonomous robots, was fascinating and exciting.
RoboCup enthusiastically and concretely introduced three robot soccer leagues, namely “Simulation,” “SmallSize,”
and “MiddleSize,” as we explain below, and organized its first
competitions at IJCAI’97 in Nagoya with a surprising number of 100 participants [RC97]. It was the beginning of what became a continously growing research community. RoboCup established
itself as a structured organization, the RoboCup Federation www.RoboCup.org, fostering annual competition events, where the scientific challenges faced by the researchers are addressed in a setting that is attractive also to the general public. The RoboCup events have become the most popular and attended in the research fields of AI and robotics, including also a technical symposium with contributions relevant to the RoboCup competitions and beyond to the general AI and robotics.

The RoboCup is a scientific challenge used to foster research in the robotics areas, which main objective consists in developing a robotic football team able to play against a human team in the year 2050. This paper describes the rules of such a competition, the actual state of the art of robotic football players in the middle size league, and describes the main characteristics to take into account in order to build such type of robots. These aspects are described and analysed in terms of further developments.

It is well known that during their learning process youngsters prefer and enjoy exciting challenges, so that they don't get bored in school. Many of those challenges are blossoming all around the world in an annual basis, and they stimulate students because they create new objectives, they allow creativity, discovering new and unique solutions and allow comparison of the work carried out by other teams. The robotics interest has been growing quickly and many schools are adopting this knowledge area due to its multidisciplinary, for being stimulating, for allowing students creativity, for being so practical and hands-on, and it technologically sounds good. Many challenges have being created in the last few years, both pedagogical and competitive, and requiring different levels of know-how. This paper describes the most important robotics challenges in terms of it main objectives and rules, the age target, its geographical localization, its average budget and the first steps to be taken for new teams. After reading this paper teachers will be able to decide which robotic challenge is more suitable for his team.

As robots become more commonplace, the tools to facilitate knowledge transfer from human to robot will be vital, especially for non-technical users. While some ongoing work considers the role of human reinforcement in intelligent algorithms, the burden of learning is often placed solely on the computer [2]. These approaches neglect the expressive capabilities of humans, especially regarding our ability to quickly refine motor skills. Thus, when designing autonomous robots that interact with humans, not only is it ...

The Project of our group, Dance Squad, in the dance category consists in a technology innovation, because we are bringing new equipments to the competitions in Brazil, two humanoid robots Bioloid Premium type A (Robotis, n.d.). We decided to use those to find a new way to dance in the competition. Using a famous song (Gun-hyung, n.d.), we would like to transmit all of the great potential that robotic has. We expect our audience to watch and enjoy our dance presentation with robots.

O número de indivíduos da terceira idade utilizando celulares é uma parcela significativa do total de usuários, entretanto faltam pesquisas voltadas para o desenvolvimento de apli-cações para smartphones com foco nos idosos. O presente trabalho tem como objetivo investigar o impacto da com-petitividade em jogos digitais para smartphones, com foco na terceira idade visando possibilitar a criação de jogos mais adequados a este público. Para isto realizou-se uma pesquisa qualitativa tendo como abordagem o estudo de caso e utili-zando a teoria fundamentada para a análise dos dados. Os resultados apresentados indicam que a competitividade, em jogos digitais, motiva os idosos e favorece a diversão.

This work presents a new approach that allows the use of cases in a case base as heuristics to speed up Multiagent Reinforcement Learning algorithms, combining Case-Based Reasoning (CBR) and Multiagent Reinforcement Learning (MRL) techniques. This approach, called Case-Based Heuristically Accelerated Multiagent Reinforcement Learning (CB-HAMRL), builds upon an emerging technique, Heuristic Accelerated Reinforcement Learning (HARL), in which RL methods are accelerated by making use of heuristic information. CB-HAMRL is a subset of MRL that makes use of a heuristic function H derived from a case base, in a Case-Based Reasoning manner. An algorithm that incorporates CBR techniques into the Heuristically Accelerated Minimax-Q is also proposed and a set of empirical evaluations were conducted in a simulator for the Littman's robot soccer domain, comparing the three solutions for this problem: MRL, HAMRL and CB-HAMRL. Experimental results show that using CB-HAMRL, the agents learn faster than using RL or HAMRL methods.

This article presents the UaiSoccer2D team, a simulation
team of robots soccer simulated at UFSJ - Federal University of São João del-Rei, MG, Brazil. Will be present a modeling strategy for use of reinforcement learning algorithm Q-learning.

This article presents the UFSJ2D Team, a team of simulated
robots soccer from UFSJ - Federal University of Sao Joao del-Rei, MG, Brazil. The main goals in this paper are to apply reinforcement learning in the optimization of decision taking and to model a strategy using fuzzy logic.

O presente trabalho apresenta uma metodologia de modelagem da técnica de aprendizado por
reforço para times de futebol de robôs 2D. A implementação da estratégia de aprendizagem consiste de quatro
etapas: denição e discretização das ações dos agentes; denição e discretização dos estados do ambiente no
qual os agentes estão inseridos; denic~ao dos valores dos reforços da tabela R, para cada par Estado (S) X Ac~ao
(A); implementac~ao no Simulador RcSoccerSim da Robocup de Futebol de Robôs. Além disso, e proposta uma
análise estatística por meio do teste T-Pareado para compreender o sistema de aprendizagem quando submetido
a competição entre outros times.

Resumo-This paper presents solutions developed by the iBots team in the RoboCup Simulation 2D category. The Agent2D framework was chosen as support of the team development. We implemented collective moves using a Fuzzy Inference System to guide the communication between agents. In addition, we developed the dynamic exchange of tactical schemes, which occurs in response to the needs of the game. Six tactical schemes have been implemented, which are adopted according to MultiLayer Perceptron Artificial Neural Network results.

A designação de "teste não paramétrico" deve-se ao facto de não ser necessário especificar a distribuição da população de onde provém a amostra (até agora, na maior parte dos casos, admitimos que tal população tinha distribuição normal ou, pelo menos, aproximadamente normal). Os métodos não paramétricos usam procedimentos que são aplicáveis independentemente da distribuição da população; quando muito, são por vezes exigidas algumas hipóteses como a de simetria ou a de continuidade da distribuição. Alguns destes métodos podem ser aplicados a dados qualitativos (relembremos que, com excepção dos intervalos de confiança e testes para proporções, todos os procedimentos estudados até agora são aplicáveis apenas a dados quantitativos). Outra situação em que os testes paramétricos sãoúteis,é aquela em que a dimensão da amostraé muito pequena e não se conhece a distribuição exacta da população.

This document presents a novel approach to control the attack behavior of a team of simulated soccer playing robot of the Robocup 2D category. The presented approach modifies the behavior of each player only when in the state "controlling the ball". The approach is based on a modified Q-Learning algorithm that implements a continuous machine learning process. After an initial learning phase, each player uses its previous experience during the simulation of the game to decide if it should dribble, pass or kick the ball to the adversary goal. Simulation results show that the proposed approach is capable of learn how to score goals, even when faced with the -champion of the previous world championship.