I attended the 2024 Conference on Decision and Control in Milan. The photo above is a panoramic view of the city from the roof of the Duomo. This year, I had one paper, which was led and presented by my former student Matthew Wallace, who recently graduated. Continue reading →

If you’ve been following the NBA playoffs, you will know that we are on the verge of a potentially historic event: the first ever reverse-sweep! The Boston Celtics will face the Miami Heat at Boston on Monday May 29th in game 7 of the Eastern Conference finals to determine which team will advance and face the Denver Nuggets in the NBA finals. The Heat won the first 3 games of the series, and then the Celtics won the following 3 games. If the Celtics win on Monday night, they will complete a “reverse sweep”. This has never happened in any best-of-seven series in NBA playoff history! But how rare is a reverse sweep, actually?

My new article titled: “The Analysis of Optimization Algorithms: A Dissipativity Approach” appeared in the June 2022 issue of the IEEE Control Systems Magazine. In the article, I explain how the behavior of iterative optimization algorithms can be viewed through the lens of energy dissipation. In mechanical systems, total energy is always conserved. So if some of that energy is dissipated as heat, what remains (kinetic plus potential) must decrease over time, and the system will eventually come to rest. Analogously, optimization algorithms can be shown to have an abstract notion of energy that dissipates over time, driving the system to come to rest (convergent behavior). In this blog post, I explain dissipativity theory and how it pertains to algorithm analysis!

It was an honor and a privilege to be one of eight invited speakers at the bi-annual NCCR Automation Symposium. The symposium was hosted in-person at ETH Zürich in May 2022.

The theme of this symposium was “Systems Theory of Algorithms”. As explained on the symposium website, many iterative algorithms in optimization, games, and learning can be viewed as dynamical systems evolving over time with inputs (real-time signals, historic data samples, or random seeds), outputs (decision variables or algorithm iterates), and uncertainties (noise or unknown problem parameters). The dynamical systems perspective on such algorithms is key in future automation, AI, and socio-technical systems, where challenges arise from the analysis and design of algorithms

Wordle is a popular daily word game that has taken the world by storm. The rules are simple: Guess the hidden 5-letter “solution word” using at most 6 guesses. Every time you make a guess, the letter tiles of your guess word are colored to provide a hint. Wordle was recently acquired by the New York Times, and remains free, though now at a new website.

What is the best first guess to use in Wordle? Is it always possible to win in 5 moves or fewer? what about 4 moves? How much harder is Hard Mode? In this article, I will answer all these questions and more!

I had the honor and privilege to be a thesis opponent for the Ph.D. defense of Martin Heyden at Lund University’s Department of Automatic Control in Sweden. The picture on the right is with Anders Rantzer, Martin Heyden, and Richard Pates (Anders and Richard were Martin’s co-advisors). During my visit to Lund, I also gave a seminar on the topic of robust control and optimization algorithms. Lund University is one of the few places in the world to have a department dedicated solely to controls! Most other universities have controls experts spread across various departments (electrical, mechanical, industrial, aerospace, etc.).

Excited to share a new manuscript, by Bryan Van Scoy and me titled “The Speed-Robustness Trade-Off for First-Order Methods with Additive Gradient Noise”. It is now available on arXiv. I also have some slides here.

Background

When solving an optimization problem, a ubiquitous approach is to use an iterative algorithm that finds progressively better candidate solutions until a satisfactory solution is obtained. In cases where the problem instances are very large, such as in machine learning applications, gradient-based iterative methods are the most popular. Think of the task of finding the point of highest elevation in a vast landscape. At each step, you measure the slope of the terrain, and move in the direction of steepest increase. Since we don’t know what the terrain shape will be ahead of time, we want algorithms that work well for a wide variety of terrains. For this reason, a popular way of evaluating algorithms is to look at their worst-case convergence rate on a given class of functions. For example, we might be solving a large linear regression problem, so we know ahead of time that the function we are optimizing is a convex quadratic. Much work has gone into finding algorithms that are “optimal” for a given class of functions. For example, we know that the Heavy Ball method is optimal for convex quadratics and the Triple Momentum Method is optimal for smooth strongly convex functions.