Stochastic Control and Its Application

A workshop dedicated to Sean Meyn's 60th birthday

Workshop at 57th IEEE Conference on Decision and Control

Miami Beach, Florida. Dec. 16, 2018

Workshop Schedule


This workshop arose out of the interest expressed by a group of collaborators and former students of Professor Sean Meyn to celebrate Sean’s 60th birthday and honor his long-lasting contributions to the field of stochastic systems, Markov processes, networks, and control theory. Professor Meyn has made fundamental contributions to the field of decision and control for over 30 years. He has led the way in establishing a comprehensive theory for the Lyapunov stability of Markov processes and controlled Markov processes including its relationship to spectral theory and large deviations. This work has had renewed applicability to communication networks and resource allocation problems. Professor Meyn has pioneered the use of stochastic approximation techniques for reinforcement learning and simulation. This work has had significant impact in other fields as well, such as machine learning and economics. Through his work in control, visualization and performance analysis of energy markets, and more recently in decentralized control of distributed energy resources for the smart grid, he continues to have an important impact on renewable energy and large-scale control of the grid.

This workshop will bring together several of Sean’s collaborators and former students to discuss a broad range of contemporary topics in different areas of systems and control theory. All the speakers in the workshop have profited from close interactions with Sean at some point in their careers and this venue will provide them with an opportunity to share with others how this experience has enriched them as researchers and educators. The main goal of these talks is to inspire a future generation of research leaders to pursue work that promotes excellence and will thus likely have a profound impact in the field.

Invited Speakers:

  1. Professor Vivek Borkar (Indian Institute of Technology, Mumbai)

    Vivek Borkar did his B.Tech. (Elec. Egg.) at IIT Bombay in 1976, M.S. (Systems and Control) at Case Western Reserve Uni. in 1977, and Ph.D. (EECS) in Uni. of California at Berkeley in 1980. He has held regular positions at the Tata Institute of Fundamental Research (TIFR) Center and Indian Institute of Science in Bengaluru, as well as TIFR, Mumbai, before joining IIT Bombay as Institute Chair Professor of Electrical Engg. in 2011. He has held visiting positions at Uni. of Twente, MIT, Uni. of Maryland at College Park, Uni. of California at Berkeley, etc. He is a Fellow of IEEE, AMS, TWAS and the science and engineering academies in India. His research interests are in stochastic control and optimization, encompassing theory, algorithms and applications.

  2. Professor Ana Busic (Inria, Paris)

    Ana Busic is a Research Scientist at Inria, Paris, and a member of the Computer Science Department at Ecole Normale Supérieure, Paris, France. She received the Ph.D. degree in Computer Science from the University of Versailles in 2007. She was a Post-Doctoral Fellow at Inria Grenoble—Rhône-Alpes and at University Paris Diderot-Paris 7. She is a member of the Laboratory of Information, Networking and Communication Sciences, a joint lab between Inria, Institut Mines-Télécom, Nokia Bell Labs, and UPMC Sorbonne Universities. Her research interests include stochastic modeling, simulation, performance evaluation and optimization, with applications to communication networks and energy systems.

  3. Professor Peter E. Caines (McGill University)

    Peter E. Caines received the BA in mathematics from Oxford University in 1967 and the PhD in systems and control theory in 1970 from Imperial College, University of London, under the supervision of David Q. Mayne. After periods as a postdoctoral researcher and faculty member at UMIST, Stanford, UC Berkeley, Toronto and Harvard, he joined McGill University, Montreal, in 1980, where he is James McGill Professor and Macdonald Chair in the Department of Electrical and Computer Engineering. In 2000, the adaptive control paper he coauthored with G. C. Goodwin and P. J. Ramadge (IEEE Transactions on Automatic Control, 1980) was recognized by the IEEE Control Systems Society as one of the 25 seminal control theory papers of the 20th century. He is a LifeFellow of the IEEE, and a Fellow of SIAM, the Institute of Mathematics and its Applications (UK) and the Canadian Institute for Advanced Research and is a member of Professional Engineers Ontario. He was elected to the Royal Society of Canada in 2003. In 2009, he received the IEEE Control Systems Society Bode Lecture Prize, and in 2012, a Queen Elizabeth II Diamond Jubilee Medal. Peter Caines is the author of Linear Stochastic Systems, John Wiley, 1988, and his research interests include stochastic, mean-field, decentralized and hybrid systems theory together with their applications in engineering, physics, economics and biology.

  4. Professor Ken Duffy (Hamilton Institute)

    Ken Duffy is a Professor and the director of Maynooth University's interdisciplinary STEM research institute, the Hamilton Institute. He holds a B.A. (mod) and Ph.D. in mathematics, both awarded by Trinity College Dublin. His research encompasses the application of probability and statistics to science and engineering. As a result of broad multidisciplinary interests, his work has been published in mathematics journals (e.g. Annals of Applied Probability, Journal of Applied Probability, Journal of Mathematical Biology), engineering journals (e.g. IEEE Transactions on Information Theory, IEEE/ACM Transactions on Networking, IEEE Transactions on Network Science and Engineering) and scientific journals (e.g. Cell, Cell Stem Cell, Nature Communications, Science). He is a co-founder of the Royal Statistical Society's Applied Probability Section (2011), co-authored a cover article of Trends in Cell Biology (September, 2012), and is a winner of a best paper award at the IEEE International Conference on Communications (2015).

  5. Professor Muriel Médard (Massachusetts Institute of Technology)

    Muriel Medard is the Cecil H. Green Professor of Electrical Engineering and Computer Science at MIT. She was previously an Assistant Professor in the Electrical and Computer Engineering Department and a member of the Coordinated Science Laboratory at the University of Illinois Urbana-Champaign. From 1995 to 1998, she was a Staff Member at MIT Lincoln Laboratory in the Optical Communications and the Advanced Networking Groups. Professor Médard received B.S. degrees in EECS and in Mathematics in 1989, a B.S. degree in Humanities in 1990, a M.S. degree in EE 1991, and a Sc D. degree in EE in 1995, all from the Massachusetts Institute of Technology (MIT), Cambridge. She has served as an Associate Editor for the Optical Communications and Networking Series of the IEEE Journal on Selected Areas in Communications, the IEEE Transactions on Information Theory, the IEEE/OSA Journal of Lightwave Technology and the OSA Journal of Optical Networking. She has served as a Guest Editor for the IEEE Journal of Lightwave Technology, the Joint special issue of the IEEE Transactions on Information Theory and the IEEE/ACM Transactions on Networking on Networking and Information Theory and the IEEE Transactions on Information Forensic and Security: Special Issue on Statistical Methods for Network Security and Forensics. She serves on the board of Governors of the IEEE Information Theory Society as well as having served as President.

  6. Professor Peter W. Glynn (Stanford University)

    Peter W. Glynn is the Thomas Ford Professor in the Department of Management Science and Engineering (MS&E) at Stanford University, and also holds a courtesy appointment in the Department of Electrical Engineering. He received his Ph.D in Operations Research from Stanford University in 1982. He then joined the faculty of the University of Wisconsin at Madison, where he held a joint appointment between the Industrial Engineering Department and Mathematics Research Center, and courtesy appointments in Computer Science and Mathematics. In 1987, he returned to Stanford, where he joined the Department of Operations Research. From 1999 to 2005, he served as Deputy Chair of the Department of Management Science and Engineering, and was Director of Stanford's Institute for Computational and Mathematical Engineering from 2006 until 2010. He served as Chair of MS&E from 2011 through 2015. He is a Fellow of INFORMS and a Fellow of the Institute of Mathematical Statistics, and was an IMS Medallion Lecturer in 1995 and INFORMS Markov Lecturer in 2014. He was co-winner of the Outstanding Publication Awards from the INFORMS Simulation Society in 1993, 2008, and 2016, was a co-winner of the Best (Biannual) Publication Award from the INFORMS Applied Probability Society in 2009, and was the co-winner of the John von Neumann Theory Prize from INFORMS in 2010. In 2012, he was elected to the National Academy of Engineering. He was Founding Editor-in-Chief of Stochastic Systems and is currently Editor-in-Chief of Journal of Applied Probability and Advances in Applied Probability. His research interests lie in simulation, computational probability, queueing theory, statistical inference for stochastic processes, and stochastic modeling.

  7. Professor Maxim Raginsky (University of Illinois at Urbana-Champaign)

    Maxim Raginsky received his B.S. and M.S. degrees in 2000 and the Ph.D. degree in 2002 from Northwestern University, all in Electrical Engineering. He has held research positions with Northwestern, the University of Illinois at Urbana-Champaign (where he was a Beckman Foundation Fellow from 2004 to 2007), and Duke University. In 2012, he has returned to the UIUC, where he is currently an Associate Professor with the Department of Electrical and Computer Engineering and the Coordinated Science Laboratory. He received NSF CAREER award in 2013. Maxim is William L. Everitt Fellow in Electrical and Computer Engineering, UIUC from 2017. He is interested in theoretical and practical aspects of information processing and decision-making in uncertain environments under resource and complexity constraints. In my research, I use tools from information theory, machine learning, game theory, and optimal/stochastic control.

  8. Professor Ioannis Kontoyiannis (University of Cambridge)

    Ioannis Kontoyiannis was born in Athens, Greece, in 1972. He received the B.Sc. degree in mathematics in 1992 from Imperial College (University of London), and in 1993 he obtained a distinction in Part III of the Cambridge University Pure Mathematics Tripos. In 1997 he received the M.S. degree in statistics, and in 1998 the Ph.D. degree in electrical engineering, both from Stanford University. Between June and December of 1995 he worked at IBM Research, on a NASA-IBM satellite image processing and compression project. From 1998 to 2001 he was an Assistant Professor with the Department of Statistics at Purdue University (and also, by courtesy, with the Departmentof Mathematics, and the School of Electrical and Computer Engineering). Between 2000 and 2005 he was an Assistant, then Associate Professor (tenured), with the Division of Applied Mathematics and with the Department of Computer Science at Brown University. Since 2005 he has been with the Department of Informatics of the Athens University of Economics and Business, where he currently a Professor. Since 2018 he has also been a Professor with the Department of Engineering of the University of Cambridge, where he holds the Chair of Information and Communications. In 2002 he was awarded the Manning endowed assistant professorship; in 2004 he was awarded the Sloan Foundation Research Fellowship; in 2005 he was awarded an honorary Master of Arts Degree Ad Eundem by Brown University; in 2009 he was awarded a two-year Marie Curie Fellowship; in 2011 he was elevated to the grade of IEEE Fellow. He has published over 40 journal articles in leading international journals and over 90 conference papers in the top international conferences in his field. He also holds two U.S. patents. He has been a plenary speaker in several conferences and has given invited lectures in many international meetings as well as in various departments in leading institutions around the world, including M.I.T., Berkeley, Stanford, Columbia University and Cambridge University. He has served on the editorial board of the American Mathematical Society's Quarterly of Applied Mathematics journal, the IEEE Transactions on Information Theory, Springer-Verlag's Acta Applicandae Mathematicae, Springer-Verlag’s Lecture Notes in Mathematics book series, and the online journal Entropy. He has served as a chair or member of the program committee of numerous IEEE conferences, and he also served a short term as Editor-in-Chief of the IEEE Transactions on Information Theory. His research interests include data compression, applied probability, information theory, statistics, and mathematical biology.

  9. Professor Roland Malhamé (Polytechnique Montréal)

    Roland Malhamé received the Bachelor’s, Master’s and Ph.D. degrees in Electrical Engineering from the American University of Beirut, the University of Houston, and the Georgia Institute of Technology in 1976, 1978 and 1983 respectively. After single year stays at University of Quebec, and CAE Electronics Ltd (Montreal), he joined in 1985 École Polytechnique de Montréal, where he is Professor of Electrical Engineering. In 1994, 2004, and 2012 he was on sabbatical leave respectively with LSS CNRS (France), École Centrale de Paris, and University of Rome Tor Vergata. His interest in statistical mechanics inspired approaches to the analysis and control of large scale systems has led him to contribute in the area of aggregate electric load modeling, and to the early developments of the theory of mean field games. His current research interests are in collective decentralized decision making schemes, and the development of mean field based control algorithms in the area of smart grids and communication systems. From June 2005 to June 2011, he headed GERAD, the Group for Research on Decision Analysis. He is an Associate Editor of International Transactions on Operations Research, and IEEE Transactions on Automatic Control.

  10. Professor Eric Moulines (Ecole Polytechnique, Paris)

    Eric Moulines received the the Engineering degree from Ecole Polytechnique, Paris, France, in 1984, the Ph. D. degree in electrical engineering from Ecole Nationale Supérieure des Télécommunications, in 1990. In 1990, he joined the Signal and Image processing department at Télécom ParisTech where he became a full professor in 1996. In 2015, he joined the Applied Mathematics Center of Ecole Polytechnique, where he is currently a professor in statistics. His areas of expertise include computational statistics, Bayesian machine learning, statistical signal processing and time-series analysis. His current research topics cover large-scale (Bayesian) inference with applications to inverse problems and machine learning and non-linear filtering. He has published more than 100 papers in leading journals of the field. In 1997 and 2006, he receive the Best paper Award of the IEEE Signal Processing Society (for papers in IEEE Trans. On SignalProcessing). He served in the editorial boards of IEEE Trans. On Signal Processing, Signal Processing, Stochastic Processes and Applications, Journal of Statistical Planning and Inference. He was the Editor-in-Chief of Bernoulli from 2013-2016. E. Moulines is an EURASIP and IMS fellow. He was the recipient of the 2010 Silver Medal from the Centre National de Recherche Scientifique and the 2011 Orange prize of the French Academy of Sciences. He was elected to the academy of sciences in 2017.

  11. Professor Matias Negrete-Pincetic (Pontificia Universidad Católica de Chile)

    Matias Negrete-Pincetic received a B.Sc. in Electrical Engineering and a M.Sc in Physics from Pontificia Universidad Catolica de Chile, and a Physics and a Ph.D. in Electrical and Computer Engineering from the University of Illinois at Urbana-Champaign, USA. He was a Postdoctoral Associate at the University of California, Berkeley, CA, USA. Currently, he is an Assistant Professor at the Electrical Engineering Department at Pontificia Universidad Catolica de Chile, co-director of the Optimization, Control and Markets Lab (OCM-Lab) and member of the academic board of the UC Energy Research Center. His current research activities include operation, control and planning of energy systems, stochastic control, electricity market design and energy policy.

  12. Professor Benjamin Van Roy (Stanford University)

    Benjamin Van Roy is a Professor of Electrical Engineering, Management Science and Engineering, and, by courtesy, Computer Science, at Stanford University, where he has served on the faculty since 1998. He has served on the editorial boards of Machine Learning, Mathematics of Operations Research, for which he co-edits the Learning Theory Area, Operations Research, for which he edited the Financial Engineering Area, and the INFORMS Journal on Optimization. He has also led research programs at and/or founded several technology companies, including Unica (acquired by IBM), Enuvis (acquired by SiRF), and Morgan Stanley. He received the SB in Computer Science and Engineering and the SM and PhD in Electrical Engineering and Computer Science, all from MIT. He has been a recipient of the MIT George C. Newton Undergraduate Laboratory Project Award, the MIT Morris J. Levin Memorial Master's Thesis Award, the MIT George M. Sprowls Doctoral Dissertation Award, the National Science Foundation CAREER Award, the Stanford Tau Beta Pi Award for Excellence in Undergraduate Teaching, and the Management Science and Engineering Department's Graduate Teaching Award. He is an INFORMS Fellow and has been a Frederick E. Terman Fellow and a David Morgenthaler II Faculty Scholar. He has held visiting positions as the Wolfgang and Helga Gaul Visiting Professor at the University of Karlsruhe and as the Chin Sophonpanich Foundation Professor and the InTouch Professor at Chulalongkorn University.

  13. Professor Venugopal V. Veeravalli (University of Illinois at Urbana-Champaign)

    Venugopal V. Veeravalli received the B.Tech. degree (Silver Medal Honors) from the Indian Institute of Technology, Bombay, in 1985, the M.S. degree from Carnegie Mellon University, Pittsburgh, PA, in 1987, and the Ph.D. degree from the University of Illinois at Urbana-Champaign, in 1992, all in electrical engineering. He joined the University of Illinois at Urbana-Champaign in 2000, where he is currently a Professor in the Department of Electrical and Computer Engineering and the Coordinated Science Laboratory. He served as a Program Director for communications research at the U.S. National Science Foundation in Arlington, VA from 2003 to 2005. He has previously held academic positions at Harvard University, Rice University, and Cornell University, and has been on sabbatical at MIT, IISc Bangalore, and Qualcomm, Inc. His research interests include distributed sensor systems and networks, wireless communications, detection and estimation theory, including quickest change detection, and information theory. Prof. Veeravalli was a Distinguished Lecturer for the IEEE Signal Processing Society during 2010-2011. He has been an Associate Editor for Detection and Estimation for the IEEE Transactions on Information Theory and for the IEEE Transactions on Wireless Communications. Among the awards he has received for research and teaching are the IEEE Browder J. Thompson Best Paper Award, the National Science Foundation CAREER Award, and the Presidential Early Career Award for Scientists and Engineers (PECASE).


  1. Professor Vivek Borkar. Stochastic approximation revisited: some new wine in old bottle.

    This talk will present some recent results concerning convergence rate of stochastic approximation algorithms and some variants and spin-offs thereof.

  2. Professor Ana Busic. Dynamic matching: approximate optimality with bounded regret.

    (Abstract will be updated soon)

  3. Professor Peter E. Caines. Systems on Unbounded Networks and the Graphon Mean Field Game Equations.

    Networks are ubiquitous in modern society and the need to analyse, design and control them is evident. However many technical and social networks apparently grow unboundedly over time. This has the undesirable consequence that, inevitably, any method founded upon techniques whose effectiveness decreases with the size of the network will eventually be overwhelmed. We present a framework called Graphon Mean Field Game (GMFG) theory for the analysis and control of non-cooperative dynamical game systems distributed over networks of unbounded size. This work is based upon the recently developed and profoundly influential graphon theory of large networks and their infinite limits. A theory for the centralized control of asymptotically infinite networks has already been formulated within the framework of dynamical systems on graphons [Gao and Caines, CDC 2017]. The current work greatly extends that analysis to populations of competing dynamical agents for which the game theoretic equilibria are expressed in terms of the newly defined Graphon Mean Field (GMFG) equations, these being a significant generalization of the classical MFG PDEs. Furthermore, existence and uniqueness theorems for GMFG equations are given together with a sketch of the corresponding epsilon-Nash theory for GMFG systems. Work with Minyi Huang

  4. Professor Ken Duffy and Professor Muriel Medard. "Meyn Control".

    (Abstract will be updated soon)

  5. Professor Peter Glynn. Approximations for Markov Chains with Non-stationary Transition Probabilities.

    (Abstract will be updated soon)

  6. Professor Maxim Raginsky. Revisiting diffusions for global optimization: Lyapunov stability and metastability.

    (Abstract will be updated soon)

  7. Professor Ioannis Kontoyiannis. Variable-dimension MCMC for Model Selection of Discrete Time Series.

    (Abstract will be updated soon)

  8. Professor Roland Malhamé. On the ergodic properties of a class of stochastic hybrid systems with applications to power and manufacturing systems.

    We consider a class of thermostatically controlled (TCL) power system loads such as air conditioners, electric space heaters and electric water heaters. The first two types of loads are modeled via hybrid systems driven by random diffusion processes, while electric water heaters are modeled via hybrid systems driven by jump Markov processes.

    It is important to understand the aggregate dynamics of such loads, as they are increasingly viewed as collectively capable of acting as virtual batteries that could help in mitigating the variability of intermittent renewable sources of energy. The simplest representations of TCL’s involves scalar temperature processes, and we review past work establishing their ergodic properties. The analysis in this simpler case can be based on classical renewal theory. When multidimensional versions of the models are considered, one needs more powerful tools based on Harris recurrence. We hope to illustrate how the sets of tools developed by Meyn and Tweedie can help address this more general modeling framework.

  9. Professor Eric Moulines. 50 Sean's recipes for approximating the solution of the Poisson equation.

    A new methodology is presented for the construction of control variates to reduce the variance of additive functionals of Markov Chain Monte Carlo (MCMC) samplers. Our control variates are defined as linear combinations of functions whose coefficients are obtained by minimizing a proxy for the asymptotic variance.

    The construction is theoretically justified by two new results. We first show that the asymptotic variance of some well-known MCMC algorithms, including the Random Walk Metropolis (RWM), the Unadjusted Langevin Algorithm (ULA) and the Metropolis Adjusted Langevin algorithm, are the limit (when a parameter goes to zero of the asymptotic variance of the Langevin diffusion (for RWM and MALA, this parameter is the variance of the proposal; for the ULA, it is the time-step of the discretization). Second, we provide an explicit representation of the optimal coefficients minimizing the asymptotic variance of the Langevin diffusion. Several examples of Bayesian inference show that the corresponding reduction in the variance is significant.

  10. Professor Matias Negrete-Pincetic. Stochastic Control Problems in Local Electricity Markets.

    (Abstract will be updated soon)

  11. Professor Benjamin Van Roy. The Trajectory to Zap-Q

    Among the many fields to which Sean has contributed is reinforcement learning. This is a topic that has been attracting a great deal of attention in recent years, and viewed by many as driving the future of artificial intelligence. Temporal-difference learning serves as a cornerstone in this area. I will discuss some history on temporal-difference learning leading up to the Zap Q-learning algorithm of Devraj and Meyn and what is special about this recent development. Time permitting, I will discuss further issues that need to be addressed in continuing this research trajectory.

  12. Professor Venugopal V. Veeravalli. Quickest Detection of Dynamically Evolving Events in Networks.

    (Abstract will be updated soon)



The workshop will consist of 2 morning sections and 2 afternoon sections, for a total of 6 hours. The specific times for the sessions will be arranged in coordination with the conference organizers.