Little Green BATS Humanoid 3D Simulation Team Description Paper Sander van Dijk, Martin Klomp, Jeroen Kuijpers, A. Bram Neijt, Matthijs Platje, Mart van de Sanden, Marijn Stollenga, and Alle Veenstra University of Groningen, Artificial Intelligence Department, The Netherlands

Abstract. The RoboCup 3D vice world champion team the Little Green BATS uses several AI methods to control and train their virtual soccer agents. A hierarchical behavior based architecture coordinates all skills. The architecture allows planning and parallel and competing behaviors. Several methods for low level movement generation are available to the agent, like setting joint angle trajectories and sinusoidal control. The latter is used to train the most critical skill of all, running, with a Genetic Algorithm.

1

Introduction

Agents that live in complex environment usually face a difficult problem: they need to show high level intelligence to survive, but the only way to directly interact with the world is through very basic senses and actions. They need a form of abstraction on these senses and actions to be able to act and response to difficult situations in real time. This abstraction has to take place at different levels, because there is no clear distinction between low level behavior and high level behavior. It is even hard to tell what kind of behavior is of a higher level than other kinds, so abstraction should be possible in any degree, at any level and over any kind of abstraction that is already made at lower levels. To account for this abstraction, we use a hierarchical behavior model in which the agent’s intelligence is constructed as a tree of behaviors, each controlling other behaviors at lower levels to supply a new abstraction in the agent’s senses and actions. The model is focussed on construction of the behavioral tree by hand using human high level knowledge, either top down or bottom up, with the ability to improve the behavior through learning. In the next section we describe the behavior model. Section 3 will cover the movement methods used by our agents.

2

BATS Hierarchical Behavior Model

The hierarchical behavior model is based on the following key ideas: – A type of behavior is defined by the type of goals it tries to achieve – A behavior can set subgoals to be achieved serially by other behaviors

– An agent has one highest level goal (e.g. ‘win the game’ for a soccer agent) – Behavior selection is done based on a behavior’s capability to reach a goal Therefore, a behavior consists of a sequence of steps. Each step has a subgoal and a set of sub behaviors that can be chosen to achieve these goals. This results in a tree of behaviors with the top level, most abstract behavior at the root and the lowest level, most primitive behaviors at the leafs. The latter behaviors don’t perform any behavior selection anymore, but only perform real world actions. At every step of a behaviors sequence, several sub goals could be achieved in parallel. Also, multiple sub behaviors can be defined for a single subgoal. These then compete for selection and the sub behavior with the highest capability of achieving the subgoal wins. The behavior tree is constructed during runtime, based on an XML configuration file. This way the agent’s architecture and parameters can be changed thoroughly but quickly, without having to recompile the agent’s source code.

3

Movement Control

The RoboCup 3D humanoid agent is controlled by setting motor joint velocities. The BATS agents have several methods for describing motion at a higher level. These descriptions are translated to joint angles, which are achieved by setting the joint angular velocities as follows: vi (t) = γ(αi′ (t) − αi (t)) ,

(1)

where vi (t) is the angular velocity for joint i at time t, γ a gain parameter and α′ is a goal angle. 3.1

Joint Angle Trajectories

The BATS agents use a very simple scripting language to define trajectories for their joints. The agent plays the script from begin to end, achieving the joint angles set in a line of script using (1) before going to the next line. It is possible to set a certain time to wait before going to the next line and to set the maximum velocity per joint. This method of movement control is used to create non-cyclical behaviors like standing up and kicking. 3.2

Sinusoidal Joint Control

A lot of humanlike movement is cyclical, most notably walking. The BATS agents generate this kind of behavior using a sinusoidal pattern generator. This generator controls joint angles using the following equation: αi′ (t) =

N X j=1

Aj sin(ωj t + θj ) + Cj .

(2)

For some of the behaviors, like the small, high frequent stepping behavior to position the agent near the ball, the parameters are set by hand. They all are based on the same sinusoidal step-in-place behavior, with an extra out-of-phase sinusoid to create forwards/backwards/sidewards stepping or turning behavior. The running behavior on the other hand is tweaked using a Genetic Algorithm. The genotype consists of the parameters Aj , ωj , θj , Cj with N = 2, for 6 joints: the hip, knee and ankle X-axis joints (i.e. LEG2, LEG4 and LEG5) of each leg. Each generation consists of 200 individuals, the first generation is initialized randomly. Tournament selection with a tournament size of 2 is used to select the next generation, after which small mutations are applied to the genotype. Good individuals are found within a few hundred generations. The running gait found this way was one of the fastest, if not the fasted gait in the 3D simulation league in Atlanta, 2007 and was the main cause of the BATS winning the Latin American Open 3D simulation league, 2007.