Nonlinear filtering
Background
Controllability and observability are foundational concepts in dynamical systems and control theory. The dual nature of these concepts is of fundamental importance for both analysis and design of algorithms.
Duality is coeval with the origin of modern control theory: The original duality principle appears in the seminal (1961) paper of Kalman-Bucy, where the problem of minimum variance estimation is shown to be dual to a linear quadratic (LQ) optimal control problem. Notably, duality explains why, with the time arrow reversed, the covariance update equation of the Kalman filter is the same as the differential Riccati equation (DRE) of optimal control.
Overview of research
For sixty years since Kalman and Bucy's seminal work, generalization of Kalman-Bucy duality to nonlinear stochastic systems (hidden Markov models) was believed to be impossible.
Not any more! Because we did the impossible. The foundational paper introduced a dual optimal control problem for nonlinear filtering. The mathematics of the paper is wild -- the constraint for the optimal control problem is a backward stochastic differential equation (BSDE). A BSDE is a particular form of stochastic dynamical system where the arrow of time runs backwards. Just like it did in that beautiful movie Tenet!
It is shown in our paper that the BSDE-based formulation is an exact extension of the original Kalman-Bucy duality -- in the sense that the dual optimal control problem has the same minimum variance structure for both linear and nonlinear filtering problems. For this paper, Jin Won Kim won the Best Student Paper Award at the IEEE Conference on Decision and Control (CDC) 2019 from a competitive field of 65 world-wide nominations for this award.
Since the foundational paper, we have written three more papers on the topic of duality:
In an MTNS paper, observability of an HMM is defined in dual terms: as controllability of the BSDE. It is shown that (i) the resulting characterization is equivalent to the definition of observability for HMMs; and (ii) the BSDE is a dual to the Zakai equation of nonlinear filtering.
The CDC paper is the first of the two papers on the subject of nonlinear filter stability (asymptotic forgetting of the initial condition). A key contribution of the paper is the notion of conditional Poincare inequality (PI) which is shown to yield filter stability. Using the dual methods, we are able to derive all the prior results where explicit convergence rates are obtained.
The CDC paper is the second of the two papers on the subject of nonlinear filter stability. The contributions of this paper are two-fold: (i) a definition is introduced for the stabilizability of the BSDE; and (ii) shown to be necessary and sufficient for filter stability. The development of the paper -- introduction of the dual system, stabilizability definition, and its use the in the filter stability analysis -- is entirely parallel to the Kalman filter stability theory.
Presentations
What is the Lagrangian for nonlinear filtering? ISS Informal Systems Seminar / Séminaire informel de théorie des systèmes, McGill University, April 2, 2021.
Slides from 58th IEEE Conference of Decision and Control, Nice, France, Dec 11-13 2019. Winner of the Best Student Paper Award.
Publications
J. W. Kim, P. G. Mehta and S. P. Meyn, "What is the Lagrangian for nonlinear filtering?" in 2019 IEEE Conference on Decision and Control (CDC), Dec 2019, pp. 1607-1614.
J. W. Kim and P. G. Mehta, "A dual characterization of observability for stochastic systems," in 24th International Symposium on Mathematical Theory of Networks and Systems (MTNS), 2021.
J. W. Kim, P. G. Mehta and S. P. Meyn, "The conditional Poincaré inequality for filter stability," in 2021 IEEE Conference on Decision and Control (CDC), Dec 2021.
J. W. Kim, P. G. Mehta and S. P. Meyn, "A dual characterization of the stability of the Wonham filter," in 2021 IEEE Conference on Decision and Control (CDC), Dec 2021.
J. W. Kim and P. G. Mehta, "A dynamic programming formulation for the nonlinear filter," in 7th Indian Control Conference, 2021.
Some useful papers on filter stability
P. Chigansky, R. Liptser, and R. Van Handel, “Intrinsic methods in filter stability,” Handbook of Nonlinear Filtering, 2009.
D. L. Ocone and E. Pardoux, “Asymptotic stability of the optimal filter with respect to its initial condition,” SIAM Journal on Control and Optimization, vol. 34, no. 1, pp. 226–243, 1996.
P. Baxendale, P. Chigansky, and R. Liptser, “Asymptotic stability of the Wonham filter: Ergodic and nonergodic signals,” SIAM Journal on Control and Optimization, vol. 43, no. 2, pp. 643–669, 2004.
R. van Handel, “Nonlinear filtering and systems theory,” in Proceedings of the 19th International Symposium on Mathematical Theory of Networks and Systems, 2010.
R. van Handel, “Observability and nonlinear filtering,” Probability theory and related fields, vol. 145, no. 1-2, pp. 35–74, 2009.
A. Budhiraja, “Asymptotic stability, ergodicity and other asymptotic properties of the nonlinear filter,” in Annales de l’IHP Probabilit´es et statistiques, vol. 39, 2003, pp. 919–941.
R. Atar and O. Zeitouni, “Exponential stability for nonlinear filtering,” in Annales de l’Institut Henri Poincare (B) Probability and Statistics, Elsevier, vol. 33, 1997, pp. 697–725.
C. McDonald and S. Yüksel, “Converse results on filter stability criteria and stochastic non-linear observability,” arXiv preprint, arXiv:1812.01772, 2020.