Partially Observable Stochastic Games — An Introduction to Plan-Time Dynamic Programming

Résumé: In our daily lives, we interact with others and make decisions that impact not only us but also those around us. It is important to recognize that these decisions can have both immediate and long-term consequences. And making them in isolation can lead to unfavorable outcomes. In fact, a decision we make today can have a ripple effect on the decisions made by others tomorrow and beyond. Ignoring this relationship can hinder us from achieving our goals. For example, a driver who recklessly crosses an intersection while another driver is entering could cause an accident, resulting in a failure to achieve the goals of both parties involved.

To handle multi-agent sequential decision-making under uncertainty, game, decentralized control, and decision theories provide the necessary foundations. Several models, principles, and algorithms have emerged in these fields to handle the increasing prevalence of multi-agent systems in society. However, instead of working together to design efficient planning, sampling, and reinforcement learning algorithms with performance guarantees, these theories have been scattered.

The purpose of this talk is to combine several theoretical foundations of intelligent agents and multi-agent systems. We aim to address partially observable stochastic games (POSGs) using a unified yet simple dynamic programming theory. By doing so, we hope to bring clarity to the field and establish a more comprehensive understanding of this complex topic. The talk will give a brief introduction to plan-time dynamic programming, a unifying methodology for sequential decision-making involving multiple agents.