Mean Field Games


Objective:

The objective here is to explore phase transition and self organization in large population dynamic systems. A central goal of this research is the largely classical question of emergent behavior in these dynamical systems:

How does collective (global) behavior depend upon the local interactions? We bring new tools to this and surrounding questions, along with new applications in engineering and biology.

Background:

In engineering applications, emergent collective behavior is almost always viewed as a bane. Take, for example, the scenes witnessed at the opening of the London's Millennium bridge.

On June 10, 2000 a bridge opened in London to mark the dawn of the new millennium. As people poured on to the Millennium bridge, it quickly became apparent that there was a problem. In the words of Pat Dallard of Arup Corporation, the firm responsible for bridge design,

''We were witnessing the bridge performing in a way that we hadn't anticipated ... We felt that we understood it very well and here it is doing something completely unexpected.''

The something unexpected referred to in Dallard's interview can be viewed on the You Tube video

The roots of the problem lay in the assumptions made in the design of the bridge. It is commonly assumed that the motion of pedestrians is incoherent. On this particular day, on this particular bridge, Pat Dallard observed that the ''motion of the crowd was synchronized with the motion of the bridge.''
As it can be seen in the video, as the bridge began to sway, the pedestrians modified their gait to maintain balance. The emergence of coherence created a lateral force on the bridge that reinforced its sway. It is this positive feedback that led to the striking oscillatory motion of the bridge that could have resulted in its collapse.

In the case of a bridge, there is no question that the emergent dynamic behavior in the form of the synchronized motion of the crowd is undesirable. We come to the same conclusion in many other contexts: For example, the power grid can show similar cascading oscillations, with similar consequences..

Motivation:

A source of inspiration for our research is the optimistic view taken in other fields, such as biology and economics.

A premise of our research is that in biological and economic systems, the emergent collective behavior is at the very core of what makes the system useful and interesting. In economics, for example, Adam Smith's invisible hand guides the smooth functioning of the free markets. Even though an individual merely seeks her own selfish gain, the invisible hand promotes the greater collective good. The invisible hand can be viewed as a self-organized collective state that emerges within a large population of self-interested individuals.

The invisible hand may well be present in the functioning of biological systems too. For instance, neural rhythms in brain circuits (e.g., the thalamocortical circuit) have been linked to information processing, communication, neuronal plasticity, memory formation, and ultimately consciousness itself.1 The self-generated (emergent) neural rhythms can be viewed as an invisible hand that guides the smooth functioning of a healthy brain. The analogy is meant to be suggestive – the invisible hand represents the collective network behavior that promotes an end greater than the activity of an individual neuron.

As the example of the Millennium bridge suggests, the collective emergent behavior need not always be desirable – the invisible hand can sometimes turn into a fist: In financial markets, liquidity can freeze up as a result of the self-reinforcing lock-step behavior of the individuals. In a recent article in The New Yorker magazine, John Cassidy makes a powerful analogy between synchronized crowd behavior in the millennium bridge and the credit-freeze experienced in the recent financial crisis.. In biology, certain forms of thalamocortical neural rhythms are a symptom of neuropathological disorder such as epilepsy.

Research Theme:

There are many approaches within the field of economics to understand the relationship between individual and collective (group) behavior. Prominent among these is the application of game theory to obtain economic equilibria based on rational actions of self-interested individuals. These equilibria may not be ideal: Even when the behavior of individuals is entirely rational from the individual's perspective, the resulting collective behavior in the marketplace may be catastrophic. The bridge provides another case in which the synchronized motion of crowd can be viewed as an example of such ''rational irrationality.'' Any individual pedestrian will naturally modify their gait as a rational strategy to avoid falling down on a swaying bridge. However, this innocent behavior of individuals collectively sustains and reinforces the (irrational) swaying of the bridge.

In the field of dynamical systems the emergent collective behavior in large population systems is regarded as a phase transition. In the Millenium bridge, the change of mode from quiescent bridge to one exhibiting violent oscillation is an example of the phase transition.

The game theoretic viewpoint is central to our study of emergent dynamics in large population dynamic systems. The phase transition in the Millennium bridge represents a particularly simple example that nonetheless captures all the main features: a collection of heterogeneous nonlinear systems (pedestrians) with selfish goals (of crossing the bridge while maintaining balance), that exhibits a phase transition from an equilibrium state (where pedestrian motion is incoherent) to a non-equilibrium state (where the pedestrian motion is synchronized). The non-equilibrium state represents an emergent dynamic behavior of the collective that sustains and reinforces the swaying of the bridge, that is not the intent of the pedestrians actions.

Results:

We have developed a methodological framework, rooted in dynamical systems theory and game theory, that relates the emergent collective dynamic behavior of the population to the underlying interaction/control mechanisms -- in particular those resulting from competitive interactions amongst individuals.

Presentations:

Institute of Systems Research (ISR), University of Maryland, College Park, March 2010.

Phase transition in large population games: An application to synchronization of coupled oscillators

Electrical Engineering Seminar, Harvard University, Boston, April 1 2011.

Mean-field methods in estimation and control, with applications to synchronization of coupled oscillators

Papers:

Yin, H., P. G. Mehta, S. P. Meyn and U. V. Shanbhag, ''Synchronization of Coupled Oscillators is a Game,'' IEEE Transactions on Automatic Control, 57:4, 920-935, April 2012.

Abstract and Downloads

The purpose of this paper is to understand phase tran-sition in noncooperative dynamic games with a large number of agents. Applications are found in neuroscience, biology, and eco-nomics, as well as traditional engineering applications.

Yin, H., P. G. Mehta, S. P. Meyn and U. V. Shanbhag, ''Bifurcation Analysis of a Heterogeneous Mean-field Oscillator Game,'' In the Proceedings of the IEEE Conference on Decision and Control, Orlando, 3895-3900, Dec 2011.

Abstract and Downloads

This paper studies the phase transition in a heterogeneous mean-field oscillator game model using methods from bifurcation theory.

Yin, H., P. G. Mehta, S. P. Meyn and U. V. Shanbhag, ''On the Efficiency of Equilibria in Mean-field Oscillator Games,'' In the Proceedings of the American Control Conference, San Francisco, 5354-5359, June 2011.

Abstract and Downloads

Here, we examine the efficiency of the associated mean-field equilibria with respect to a related welfare optimization problem. We construct variational problems both for the noncooperative game and its centralized counterpart and employ these problems as a vehicle for conducting this analysis. Using a bifurcation analysis, we analyze the variational solutions and the associated efficiency loss. An expression for the efficiency loss is obtained

Yin, H, P. G. Mehta, S. P. Meyn and U. V. Shanbhag. ''Learning in Mean-field Oscillator Games,'' In the Proceedings of the IEEE Conference on Decision and Control, Atlanta, 3125-3132, Dec 2010.

Abstract and Downloads

This research concerns a noncooperative dynamic game with large number of oscillators. The states are inter-preted as the phase angles for a collection of non-homogeneous oscillators, and in this way the model may be regarded as an extension of the classical coupled oscillator model of Kuramoto.

Deng, K., P. Barooah and P. G. Mehta. ''Mean-field Control for Energy Efficient Buildings,'' In the Proceedings of the American Control Conference, Montreal, 3044-3049, June 2012.

Abstract and Downloads

In this paper, we consider the problem of dis-tributed set-point temperature regulation in a large building. With a large number of zones, the problem becomes intractablewith standard control approaches due to the large state space dimension of the dynamic model. To mitigate complexity, we develop here a mean-field control approach applicable to large-scale control problems in buildings. The mean-field here represents the net effect of the entire building envelope on any individual zone.

Awards:

Paper ''Synchronization of coupled oscillators is a game'' with graduate student, Huibing Yin, finalist for the Best Student Paper Award at the American Control Conference, Baltimore, June 2010

Acknowledgements:

This work was supported by the Computational Science and Engineering fellowship at the University of Illinois and by the Department of Energy grant 2008-05542.