Learning to Co-operate in Multi-Agent Systems Experiments with the RoboCup Simulator

In recent years, two major areas of computer science have started converging. Artificial intelligence research is moving towards realistic domains requiring real-time responses, and real-time systems are moving towards more complex applications requiring intelligent behaviour. This book addresses the question of whether agents can learn to become individually skilled and also learn to co-operate in the presence of both teammates and adversaries in a complex, real-time, noisy environment with no communication. To answer this question this work starts by presenting a multi-threaded agent architecture capable of dealing with the logical and timing challenges of such an environment. The decision making process is broken down into simple modules that link together an agents perception to its actions. The book demonstrates how a sparse distributed memory model can be used as a generalisation component for tasks that involve large state spaces. It further demonstrates how reinforcement learning can be linked to such a memory model and produce intelligent action. Experimental results demonstrate how a learned policy can outperform fixed, hand-coded ones.