ITS Berkeley

Lagrangian Control through Deep-RL: Applications to Bottleneck Decongestion

Vinitsky, Eugene
Parvate, Kanaad
Kreidieh, Aboudy
Wu, Cathy
Bayen, Alexandre
2018

Using deep reinforcement learning, we derive novel control policies for autonomous vehicles to improve the throughput of a bottleneck modeled after the San Francisco-Oakland Bay Bridge. Using Flow, a new library for applying deep reinforcement learning to traffic micro-simulators, we consider the problem of improving the throughput of a traffic benchmark: a two-stage bottleneck where four lanes reduce to two and then reduce to one. We first characterize the inflow-outflow curve of this bottleneck without any control. We introduce an inflow of autonomous vehicles with the intent of...

A Methodology for Evaluating the Performance of Model-Based Traffic Prediction Systems

Gomes, Gabriel
Gan, Qijian
Bayen, Alexandre M.
2018

Model-based traffic prediction systems (mbTPS) are a central component of the decision support and ICM (integrated corridor management) systems currently used in several large urban traffic management centers. These models are intended to generate real-time predictions of the system’s response to candidate operational interventions. They must therefore be kept calibrated and trustworthy. The methodologies currently available for tracking the validity of a mbTPS have been adapted from approaches originally designed for off-line operational planning models. These approaches are insensitive...

Time-Continuous Instantaneous and Past Memory Routing on Traffic Networks: A Mathematical Analysis on the Basis of the Link-Delay Model

Bayen, Alexandre
Keimer, Alexander
Porter, Emily
Spinola, Michele
2019

The problem of routing messages along near-shortest paths in a distributed network without using complete routing tables is considered. It is assumed that the nodes of the network can be assigned suitable short names at the time the network is established. Two space-efficient near-shortest-path routing schemes are given for the class of planar networks. Both schemes use the separator property of planar networks in assigning the node names and performing the routings. For an n-node network, the first scheme uses $O(\log n)$-bit names and a total of $O(n^{{4 / 3}} )$ items of routing...

Boundary Observer for Congested Freeway Traffic State Estimation via Aw-Rascle-Zhang model

Yu, Huan
Bayen, Alexandre M.
Krstic, Miroslav
2019

This paper develops boundary observer for estimation of congested freeway traffic states based on Aw-Rascle-Zhang(ARZ) partial differential equations (PDE) model. Traffic state estimation refers to acquisition of traffic state information from partially observed traffic data. This problem is relevant for freeway due to its limited accessibility to real-time traffic information. We propose a boundary observer design so that estimates of aggregated traffic states in a freeway segment are obtained simply from boundary measurement of flow and velocity. The macroscopic traffic dynamics is...

An Equitable and Integrated Approach to Paying for Roads in a Time of Rapid Change

Bayen, Alexandre
Shaheen, Susan
Forscher, Edward H.
Lazarus, Jessica
2019

A brief overview of transportation user fees (historically and in a contemporary context) is presented followed by a discussion on how segmenting travel into three categories –long haul, the last mile, and at the curb–creates a new typology for transportation pricing and access mechanisms....

Simulation to Scaled City: Zero-Shot Policy Transfer for Traffic Control via Autonomous Vehicles

Jang, Kathy
Vinitsky, Eugene
Chalaki, Behdad
Remer, Ben
Beaver, Logan E.
Malikopoulos, Andreas A.
Bayen, Alexandre
2019

Using deep reinforcement learning, we successfully train a set of two autonomous vehicles to lead a fleet of vehicles onto a round-about and then transfer this policy from simulation to a scaled city without fine-tuning. We use Flow, a library for deep reinforcement learning in microsimulators, to train two policies, (1) a policy with noise injected into the state and action space and (2) a policy without any injected noise. In simulation, the autonomous vehicles learn an emergent metering behavior for both policies which allows smooth merging. We then directly transfer this policy without...

Backstepping-Based Time-Gap Regulation for Platoons

Chou, Fang-Chieh
Tang, Shu-Xia
Lu, Xiao-Yun
Bayen, Alexandre
2019

The time-gap regulation problem for a cascaded system consisting of platooned automated vehicles following a leading non-automated vehicle is investigated in this article. Under the assumption of uniform boundedness of the acceleration of the leading vehicle, a control design scheme is proposed via an extension of integral backstepping control method, where additional terms that counter the impact due to the speed change of the non-automated vehicle are used. Each automated vehicle is actuated by one backstepping controller, demonstrated by a recursive control design procedure based on...

Integrated Offline and Online Optimization-Based Control in a Base-Parallel Architecture

Jamshidnejad, Anahita
Gomes, Gabriel
Bayen, Alexandre M.
Schutter, Bart De
2019

We propose an integrated control architecture to address the gap that currently exists for efficient real-time implementation of MPC-based control approaches for highly nonlinear systems with fast dynamics and a large number of control constraints. The proposed architecture contains two types of controllers: base controllers that are tuned or optimized offline, and parallel controllers that solve an optimization-based control problem online. The control inputs computed by the base controllers provide starting points for the optimization problem of the parallel controllers, which operate in...

Regrets in Routing Networks: Measuring the Impact of Routing Apps in Traffic

Cabannes, Theophile
Sangiovanni, Marco
Keimer, Alexander
Bayen, Alexandre M.
2019

The impact of the recent increase in routing apps usage on road traffic remains uncertain to this day. The article introduces, for the first time, a criterion to evaluate a distance between an observed state of traffic and the user equilibrium of the traffic assignment: the average marginal regret. The average marginal regret provides a quantitative measure of the impact of routing apps on traffic using only link flows, link travel times, and travel demand. In non-atomic routing games (or static traffic assignment models), the average marginal regret is a measure of selfish drivers’...

Deep Truck : A Deep Neural Network Model for Longitudinal Dynamics of Heavy Duty Trucks

Albeaik, Saleh
Chou, Fang-Chieh
Lu, Xiao-Yun
Bayen, Alexandre M.
2019

This article demonstrates the use of deep neural networks (NN) and deep reinforcement learning (deep-RL) for modeling and control of longitudinal heavy duty truck dynamics. Instead of explicit use of analytical model derived information or parameters about the truck, the deep NN model is fitted to data using a brief set of historical data collected from an arbitrary driving cycle. The deep model is used in this article to design a cruise controller for the truck using model-free deep-RL. The deep model and the control loop performances are demonstrated both using state-of-the-art...