Traffic Operations and Management

Lagrangian Control through Deep-RL: Applications to Bottleneck Decongestion

Vinitsky, Eugene
Parvate, Kanaad
Kreidieh, Aboudy
Wu, Cathy
Alexandre Bayen
2018

Using deep reinforcement learning, we derive novel control policies for autonomous vehicles to improve the throughput of a bottleneck modeled after the San Francisco-Oakland Bay Bridge. Using Flow, a new library for applying deep reinforcement learning to traffic micro-simulators, we consider the problem of improving the throughput of a traffic benchmark: a two-stage bottleneck where four lanes reduce to two and then reduce to one. We first characterize the inflow-outflow curve of this bottleneck without any control. We introduce an inflow of autonomous vehicles with the intent of...

Dissipating Stop-and-Go Waves in Closed and Open Networks via Deep Reinforcement Learning

Kreidieh, Abdul Rahman
Wu, Cathy
Alexandre Bayen
2018

This article demonstrates the ability for model-free reinforcement learning (RL) techniques to generate traffic control strategies for connected and automated vehicles (CAVs) in various network geometries. This method is demonstrated to achieve near complete wave dissipation in a straight open road network with only 10% CAV penetration, while penetration rates as low as 2.5% are revealed to contribute greatly to reductions in the frequency and magnitude of formed waves. Moreover, a study of controllers generated in closed network scenarios exhibiting otherwise similar densities and...

Boundary Observer for Congested Freeway Traffic State Estimation via Aw-Rascle-Zhang model

Yu, Huan
Alexandre Bayen
Krstic, Miroslav
2019

This paper develops boundary observer for estimation of congested freeway traffic states based on Aw-Rascle-Zhang(ARZ) partial differential equations (PDE) model. Traffic state estimation refers to acquisition of traffic state information from partially observed traffic data. This problem is relevant for freeway due to its limited accessibility to real-time traffic information. We propose a boundary observer design so that estimates of aggregated traffic states in a freeway segment are obtained simply from boundary measurement of flow and velocity. The macroscopic traffic dynamics is...

Time-Continuous Instantaneous and Past Memory Routing on Traffic Networks: A Mathematical Analysis on the Basis of the Link-Delay Model

Alexandre Bayen
Keimer, Alexander
Porter, Emily
Spinola, Michele
2019

The problem of routing messages along near-shortest paths in a distributed network without using complete routing tables is considered. It is assumed that the nodes of the network can be assigned suitable short names at the time the network is established. Two space-efficient near-shortest-path routing schemes are given for the class of planar networks. Both schemes use the separator property of planar networks in assigning the node names and performing the routings. For an n-node network, the first scheme uses $O(\log n)$-bit names and a total of $O(n^{{4 / 3}} )$ items of routing...

Simulation to Scaled City: Zero-Shot Policy Transfer for Traffic Control via Autonomous Vehicles

Jang, Kathy
Vinitsky, Eugene
Chalaki, Behdad
Remer, Ben
Beaver, Logan E.
Malikopoulos, Andreas A.
Alexandre Bayen
2019

Using deep reinforcement learning, we successfully train a set of two autonomous vehicles to lead a fleet of vehicles onto a round-about and then transfer this policy from simulation to a scaled city without fine-tuning. We use Flow, a library for deep reinforcement learning in microsimulators, to train two policies, (1) a policy with noise injected into the state and action space and (2) a policy without any injected noise. In simulation, the autonomous vehicles learn an emergent metering behavior for both policies which allows smooth merging. We then directly transfer this policy without...

Regrets in Routing Networks: Measuring the Impact of Routing Apps in Traffic

Cabannes, Theophile
Sangiovanni, Marco
Keimer, Alexander
Alexandre Bayen
2019

The impact of the recent increase in routing apps usage on road traffic remains uncertain to this day. The article introduces, for the first time, a criterion to evaluate a distance between an observed state of traffic and the user equilibrium of the traffic assignment: the average marginal regret. The average marginal regret provides a quantitative measure of the impact of routing apps on traffic using only link flows, link travel times, and travel demand. In non-atomic routing games (or static traffic assignment models), the average marginal regret is a measure of selfish drivers’...

Routing on Traffic Networks Incorporating Past Memory up to Real-Time Information on the Network State

Keimer, Alexander
Alexandre Bayen
2020

In this review, we discuss routing algorithms for the dynamic traffic assignment (DTA) problem that assigns traffic flow in a given road network as realistically as possible. We present a new class of so-called routing operators that route traffic flow at intersections based on either real-time information about the status of the network or historical data. These routing operators thus cover the distribution of traffic flow at all possible intersections. To model traffic flow on the links, we use a well-known macroscopic ordinary delay differential equation. We prove the existence and...

Zero-Shot Autonomous Vehicle Policy Transfer: From Simulation to Real-World via Adversarial Learning

Chalaki, Behdad
Beaver, Logan E.
Remer, Ben
Jang, Kathy
Vinitsky, Eugene
Alexandre Bayen
Malikopoulos, Andreas A.
2020

In this article, we demonstrate a zero-shot transfer of an autonomous driving policy from simulation to University of Delaware's scaled smart city with adversarial multi-agent reinforcement learning, in which an adversary attempts to decrease the net reward by perturbing both the inputs and outputs of the autonomous vehicles during training. We train the autonomous vehicles to coordinate with each other while crossing a roundabout in the presence of an adversary in simulation. The adversarial policy successfully reproduces the simulated behavior and incidentally outperforms, in terms of...

Optimizing Mixed Autonomy Traffic Flow with Decentralized Autonomous Vehicles and Multi-Agent RL

Vinitsky, Eugene
Lichtle, Nathan
Parvate, Kanaad
Alexandre Bayen
2020

We study the ability of autonomous vehicles to improve the throughput of a bottleneck using a fully decentralized control scheme in a mixed autonomy setting. We consider the problem of improving the throughput of a scaled model of the San Francisco-Oakland Bay Bridge: a two-stage bottleneck where four lanes reduce to two and then reduce to one. Although there is extensive work examining variants of bottleneck control in a centralized setting, there is less study of the challenging multi-agent setting where the large number of interacting AVs leads to significant optimization difficulties...

Continual Learning of Microscopic Traffic Models Using Neural Networks

Farid, Yashar Zeinali
Kreidieh, Abdul Rahman
Khalighi, Farnoush
Lobel, Hans
Alexandre Bayen
2021

In a mixed-autonomy traffic scenario, where human drivers and autonomous vehicles share the streets, self-driving cars need to be able to predict in a robust manner the behaviour of human-driven vehicles, in order to guarantee a safe and smooth driving experience. Although traffic theory provides several models of human drivers, these models are often parameterized by few parameters which can limit their performance in modeling complex behaviors. The lack of sufficient model capacity and the behavioral shifts in human driving reduces the usefulness of these methods in real-life...