As Sian (1991) has pointed out many real world problems are best modelled using a set of cooperating intelligent systems (agents). There are many reasons that we could give to justify our position. First, our society consists of many interacting entities and so if we are interested to model some aspects of our society, our model needs to be structured. Also, as data often originates at different physical locations, centralized solutions are often inapplicable or inconvenient. Recent work in the field of Distributed Artificial Intelligence (DAI) and multi-agent systems (Huhns, 1987; Bond and Gasser, 1988; Durfee et al., 1989; Demazeau et al., 1991, etc.) has addressed the issues of organization, coordination and cooperation. The problems of multi-agent learning has, however, been largely ignored. One purpose of this paper is to address some issues that arise when studying ML in multi-agent systems.
Two rather different questions can be formulated in this context. First, how can multi-agent systems benefit from machine learning. Second, how can machine learning benefit from considering multi-agent set-up. As multi-agent systems are by nature complex, machine learning techniques may be the only way to achieve a robust and versatile system. The advantages of ML cannot be taken for granted, but rather have to be demonstrated in terms of its effects on cost, time, resources and product quality. One may envisage advantages defined in terms of ease of programming, maintenance, scope of application, efficiency and coordination of activity.
One may wonder why the researchers in machine learning should venture into an area so difficult as distributed AI. We believe that multi-agent learning will touch upon some of the fundamental issues of intelligence and learning that can be only understood in this context. Although communication seems to play an important role in human learning, so far this has not been studied much in ML.
Studying multi-agent learning may help us to design systems that are faster, thanks to the possibility of parallelism. Furthermore, the systems may become more robust when compared to single-agent systems. As has been shown by various authors (e.g. Gams, 1989; Buntine, 1989; Brazdil and Torgo, 1990 etc.) cross checking of results between different methods provides more reliable results.
The study of multi-agent learning poses new questions that need to be answered. For example, when should the individual systems cooperate and how. The purpose of this paper is to discuss several different approaches that have been taken. This discussion will not attempt to be exhaustive, but rather concentrate mainly on the work done in this area by the authors of this paper. However, an attempt will be made to present this work in a unified perspective and suggest directions for further work.
The paper is organized as follows. In Section 2 we shall briefly discuss autonomous agent learning. Section 3 will be dedicated to multi-agent learning. It will describe certain criteria that we can use when comparing different systems. This section describes several existing systems and approaches and is mainly oriented towards some earlier work done by the authors. The last section will discuss new horizons and future work.