ITS Berkeley

Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms

Yu, Chao
Velu, Akash
Vinitsky, Eugene
Wang, Yu
Bayen, Alexandre
Wu, Yi
2020

We benchmark commonly used multi-agent deep reinforcement learning (MARL) algorithms on a variety of cooperative multi-agent games. While there has been significant innovation in MARL algorithms, algorithms tend to be tested and tuned on a single domain and their average performance across multiple domains is less characterized. Furthermore, since the hyperparameters of the algorithms are carefully tuned to the task of interest, it is unclear whether hyperparameters can easily be found that allow the algorithm to be repurposed for other cooperative tasks with different reward structure and...

Optimizing Mixed Autonomy Traffic Flow With Decentralized Autonomous Vehicles and Multi-Agent RL

Vinitsky, Eugene
Lichtle, Nathan
Parvate, Kanaad
Bayen, Alexandre
2020

We study the ability of autonomous vehicles to improve the throughput of a bottleneck using a fully decentralized control scheme in a mixed autonomy setting. We consider the problem of improving the throughput of a scaled model of the San Francisco-Oakland Bay Bridge: a two-stage bottleneck where four lanes reduce to two and then reduce to one. Although there is extensive work examining variants of bottleneck control in a centralized setting, there is less study of the challenging multi-agent setting where the large number of interacting AVs leads to significant optimization difficulties...

Hamilton-Jacobi Formulation for State-Constrained Optimal Control and Zero-Sum Game Problems

Lee, Donggun
Keimer, Alexander
Bayen, Alexandre M.
Tomlin, Claire J.
2020

This paper presents a general Hamilton-Jacobi (HJ) framework for optimal control and two-player zero-sum game problems, both with state constraints. In the optimal control problem, a control signal and terminal time are determined to minimize the given cost and satisfy the state constraints. In the game problem, the two players interact via the system dynamics. Here, a strategy for each player, as well as a terminal time, are determined so that player A minimizes the cost and satisfies the state constraints while player B tries to prevent the success of player A. Dynamics, costs, and state...

ResiliNet: Failure-Resilient Inference in Distributed Neural Networks

Yousefpour, Ashkan
Nguyen, Brian Q.
Devic, Siddartha
Wang, Guanhua
Bayen, Alexandre
2020

Federated Learning aims to train distributed deep models without sharing the raw data with the centralized server. Similarly, in distributed inference of neural networks, by partitioning the network and distributing it across several physical nodes, activations and gradients are exchanged between physical nodes, rather than raw data. Nevertheless, when a neural network is partitioned and distributed among physical nodes, failure of physical nodes causes the failure of the neural units that are placed on those nodes, which results in a significant performance drop. Current approaches focus...

A Graph Convolutional Network with Signal Phasing Information for Arterial Traffic Prediction

Chan, Victor
Gan, Qijian
Bayen, Alexandre
2020

Accurate and reliable prediction of traffic measurements plays a crucial role in the development of modern intelligent transportation systems. Due to more complex road geometries and the presence of signal control, arterial traffic prediction is a level above freeway traffic prediction. Many existing studies on arterial traffic prediction only consider temporal measurements of flow and occupancy from loop sensors and neglect the rich spatial relationships between upstream and downstream detectors. As a result, they often suffer large prediction errors, especially for long horizons. We fill...

Continual Learning of Microscopic Traffic Models Using Neural Networks

Farid, Yashar Zeinali
Kreidieh, Abdul Rahman
Khalighi, Farnoush
Lobel, Hans
Bayen, Alexandre M.
2021

In a mixed-autonomy traffic scenario, where human drivers and autonomous vehicles share the streets, self-driving cars need to be able to predict in a robust manner the behaviour of human-driven vehicles, in order to guarantee a safe and smooth driving experience. Although traffic theory provides several models of human drivers, these models are often parameterized by few parameters which can limit their performance in modeling complex behaviors. The lack of sufficient model capacity and the behavioral shifts in human driving reduces the usefulness of these methods in real-life...

To Pool or Not to Pool? Understanding the Time and Price Tradeoffs of OnDemand Ride Users – Opportunities, Challenges, and Social Equity Considerations for Policies to Promote Shared-Ride Services

Shaheen, Susan
Lazarus, Jessica
Caicedo, Juan
Bayen, Alexandre
2021

On-demand mobility services including transportation network companies (also known as ridesourcing and ridehailing) like Lyft and Uber are changing the way that people travel by providing dynamic mobility that can supplement public transit and personal-vehicle use. However, TNC services have been found to contribute to increasing vehicle mileage, traffic congestion, and greenhouse gas emissions. Pooling rides ⎯ sharing a vehicle by multiple passengers to complete journeys of similar origin and destination ⎯ can increase the average vehicle occupancy of TNC trips and thus mitigate some of...

Reachability Analysis for FollowerStopper: Safety Analysis and Experimental Results

Chou, Fang-Chieh
Gibson, Marsalis
Bhadani, Rahul
Bayen, Alexandre M.
Sprinkle, Jonathan M.
2021

Motivated by earlier work and the developer of a new algorithm, the FollowerStopper, this article uses reachability analysis to verify the safety of the FollowerStopper algorithm, which is a controller designed for dampening stop-and-go traffic waves. With more than 1100 miles of driving data collected by our physical platform, we validate our analysis results by comparing it to human driving behaviors. The FollowerStopper controller has been demonstrated to dampen stop-and-go traffic waves at low speed, but previous analysis on its relative safety has been limited to upper and lower...

PDE Traffic Observer Validated on Freeway Data

Yu, Huan
Gan, Qijian
Bayen, Alexandre
Krstic, Miroslav
2021

This article develops a boundary observer for the estimation of congested freeway traffic states based on the Aw-Rascle-Zhang (ARZ) partial differential equations (PDEs) model. Traffic state estimation refers to the acquisition of traffic state information from partially observed traffic data. This problem is relevant for freeway due to its limited accessibility to real-time traffic information. We propose a model-driven approach in which the estimation of aggregated traffic states in a freeway segment is obtained simply from the boundary measurement of flow and velocity without knowledge...

Quasi-Dynamic Traffic Assignment using High Performance Computing

Chan, Cy
Kuncheria, Anu
Zhao, Bingyu
Cabannes, Theophile
Keimer, Alexander
Bayen, Alexandre
2021

Traffic assignment methods are some of the key approaches used to model flow patterns that arise in transportation networks. Since static traffic assignment does not have a notion of time, it is not designed to represent temporal dynamics that arise as vehicles flow through the network and demand varies through the day. Dynamic traffic assignment methods attempt to resolve these issues, but require significant computational resources if modeling urban-scale regions (on the order of millions of links and vehicles) and often take days of compute time to complete. The focus of this work is...