Morning Session (ROOM 511a)

8:30 – 8:40 Introduction [slides]
8:40 – 9:20 Roderich Gross (University of Sheffield) [slides]
    “Less is more? New approaches for swarm control and inference”
9:20 – 10:00 Karl Tuyls (University of Liverpool) [slides]
    “Towards a Unification of Paradigmatic Realizations of Multi-Agent Learning”

10:00 – 10:30 Coffee break

10:30 – 11:10 Michael L. Littman (Brown University) [slides]
    “Coordination and Joint Intention”
11:10 – 11:30 Diederik M. Roijers (Contributed talk) [slides]
    “Variational Multi-Objective Coordination”
11:30 – Poster session 1

Afternoon Session (ROOM 511a)

14:30 – 15:10 Michael Bowling (University of Alberta) [slides]
    “von Neumann’s Dream”
15:10 – 15:30 Miao Liu (Contributed talk) [slides]
    “Policy Based Reinforcement Learning in DEC-POMDPs with Bayesian Nonparametrics”
15:30 – 15:50 Charles J. M. Mathy (Contributed talk)
    “SPARTA: Fast global planning of collision-avoiding robot trajectories”
15:50 – Poster session 2

16:00 – 16:30 Coffee break

16:30 – 17:10 Frans Oliehoek (University of Amsterdam) [slides]
    “Multiagent Learning, Planning & Influences”
17:10 – 17:50 Vito Trianni (Italian National Research Council) [slides]
    “Collective Decisions: Where Swarms Dare”
17:50 – 18:20 Panel discussion
18:20 – Concluding remarks


  • Boosting Gradient Algorithms for Multi-Agent Games.
    Clemens Rosenbaum and Sridhar Mahadevan

Invited Talks

Roderich Gross (University of Sheffield)
    “Less is more? New approaches for swarm control and inference”

Robot swarms are often said to exhibit emergent properties. Yet, it is possible to design controllers with predictable outcome. We illustrate this for two canonical problems, multi-robot rendezvous and cooperative transport. The simplicity of the controllers (some do not even require arithmetic computation) facilitates their analysis. In the second part of the talk, we address the problem of inferring the rules of swarming agents through observation. We propose Turing Learning – the first system identification method not to rely on pre-defined metrics – and test it on a physical swarm of robots. Finally, we discuss novel development tools. We present OpenSwarm, an operating system for miniature robots, and formal methods for automatic code generation. We report on experiments with up to 600 physical robots.

Karl Tuyls (University of Liverpool)
    “Towards a Unification of Paradigmatic Realizations of Multi-Agent Learning”

Many real-world scenarios can be modelled as multi-agent systems, in which multiple autonomous decision makers interact in a single environment. The complex and dynamic nature of such interactions prevents hand-crafting solutions for all possible scenarios, hence learning is crucial. There exist several different classes or paradigms that address learning in multiagent systems, which we identify as individual learning (e.g. reinforcement learning), population learning (e.g. co-evolutionary learning) and protocol learning (e.g. learnable mechanism design). Although historically these classes come from very different perspectives (e.g. reinforcement learning vs swarm learning) we find that ultimately there are many commonalities among these algorithms. In this talk we aim to emphasize and unify some of these commonalities. Additionally, we discuss Evolutionary Game Theory as a tool or means to capture the dynamics of multiagent learning across the different classes of multiagent learning, which allows us to identify some of the similarities and differences.

Michael L. Littman (Brown University)
    “Coordination and Joint Intention”

We are exploring the question of how to make decisions in two-agent environments, specifically when the players’ best decisions are interdependent. In these games, players need to somehow negotiate to identify their objectives and how to jointly achieve them. We examine several different algorithms and compare them to what we observe people doing in the same two-agent games. We propose an approach to behavior coordination that uses shared experience to identify a kind of “norm” for guiding future joint action choices.

Michael Bowling (University of Alberta)
    “von Neumann’s Dream”

At the very beginning of computing and artificial intelligence, John von Neumann dreamt about the very topic of this workshop. “Real life consists of bluffing, of little tactics of deception, of asking yourself what is the other man going to think I mean to do. And that is what games are about in my theory.” And the particular game von Neumann was hinting at was poker, which played a foundational role in his formalization of game theory. Shortly after launching the field of game theory, he practically abandoned his new discipline to focus on the budding field of computing. He saw computers as the way to make his mathematics workable. Now, over 70 years later with both significant advances in computing and game theoretic algorithms, von Neumann’s dream is now a reality. Heads-up limit Texas hold’em poker, the smallest variant of poker played by humans, is essentially solved. In this talk, I will discuss how we accomplished this landmark result, along with the substantial scientific advances in our failed attempts along the way.

Frans Oliehoek (University of Amsterdam)
    “Multiagent Learning, Planning & Influences”

Multiagent systems (MASs) hold great promise as they potentially offer more efficient, robust and reliable solutions to a great variety of real-world problems. Consequently, multiagent planning and learning have been important research topics in artificial intelligence for nearly two decades. When talking about learning, however, relatively little research has addressed stochastic, partially observable environments where agents need to act based on only their individual observations only and planning for such settings remains burdened by the curse of dimensionality. In this talk, I will give an overview of some approaches to multiagent learning and planning that I have pursued in recent years. A common thread in these is the attempt to capture the locality in the way that agents may influence one another. Formalizations of such influences have lead to vast improvements in planning tractability, and I will argue that they will be critical to the advancement of multiagent learning too.

 Vito Trianni (Italian National Research Council)
    “Collective Decisions: Where Swarms Dare”

The ability of swarms to collectively choose the best option among a set of alternatives is remarkable. In particular, previous studies of the nest-site selection behaviour of honeybees have described the mechanisms that can be employed to adaptively switch between deliberate and greedy choices, the latter being taken when the value of the available alternatives is comparable. In this talk, I will review evidence about self-organised mechanisms for collective choices, and will introduce a design methodology for decentralised multi-agent systems that guarantees the attainment of desired macroscopic properties. In particular, I will present a design pattern for collective decision making that provides formal guidelines for the microscopic implementation of collective decisions to quantitatively match the macroscopic predictions. Additionally, I will provide examples of the design methodology through several case studies that showcase the viability of the approach. The case studies cover abstract multi-agent models, as well as applications in swarm robotics and cognitive radio networks.