ITS Berkeley

In this review, we discuss routing algorithms for the dynamic traffic assignment (DTA) problem that assigns traffic flow in a given road network as realistically as possible. We present a new class of so-called routing operators that route traffic flow at intersections based on either real-time information about the status of the network or historical data. These routing operators thus cover the distribution of traffic flow at all possible intersections. To model traffic flow on the links, we use a well-known macroscopic ordinary delay differential equation. We prove the existence and...

Reinforcement Learning (RL) is an effective tool for controller design but can struggle with issues of robustness, failing catastrophically when the underlying system dynamics are perturbed. The Robust RL formulation tackles this by adding worst-case adversarial noise to the dynamics and constructing the noise distribution as the solution to a zero-sum minimax game. However, existing work on learning solutions to the Robust RL formulation has primarily focused on training a single RL agent against a single adversary. In this work, we demonstrate that using a single adversary does not...

We benchmark commonly used multi-agent deep reinforcement learning (MARL) algorithms on a variety of cooperative multi-agent games. While there has been significant innovation in MARL algorithms, algorithms tend to be tested and tuned on a single domain and their average performance across multiple domains is less characterized. Furthermore, since the hyperparameters of the algorithms are carefully tuned to the task of interest, it is unclear whether hyperparameters can easily be found that allow the algorithm to be repurposed for other cooperative tasks with different reward structure and...

This paper presents a general Hamilton-Jacobi (HJ) framework for optimal control and two-player zero-sum game problems, both with state constraints. In the optimal control problem, a control signal and terminal time are determined to minimize the given cost and satisfy the state constraints. In the game problem, the two players interact via the system dynamics. Here, a strategy for each player, as well as a terminal time, are determined so that player A minimizes the cost and satisfies the state constraints while player B tries to prevent the success of player A. Dynamics, costs, and state...