ITS Berkeley

Reinforcement Learning-Based Oscillation Dampening: Scaling Up Single-Agent Reinforcement Learning Algorithms to a 100-Autonomous-Vehicle Highway Field Operational Test

Jang, Kathy
Lichtle, Nathan
Vinitsky, Eugene
Shah, Adit
Bunting, Matthew
Nice, Matthew
Piccoli, Benedetto
Seibold, Benjamin
Work, Daniel B.
Delle Monache, Maria Laura
Sprinkle, Jonathan
Lee, Jonathan W.
Bayen, Alexandre M.
2025

In this article, we explore the technical details of the reinforcement learning (RL) algorithms that were deployed in the largest field test of automated vehicles designed to smooth traffic flow in history as of 2023, uncovering the challenges and breakthroughs that come with developing RL controllers for automated vehicles. We delve into the fundamental concepts behind RL algorithms and their application in the context of self-driving cars, discussing the developmental process from simulation to deployment in detail, from designing simulators to reward function shaping. We present the...

Second-Order Time to Collision With Non-Static Acceleration

Matin, Hossein Nick Zinat
Yeo, Yuneil
Ngo, Amelie Ju-Kang
Paiva, Antonio R.
Utke, Jean
Monache, Maria Laura Delle
2025

We propose a second-order time to collision (TTC) considering non-static acceleration and turning with realistic assumptions. This is equivalent to considering that the steering wheel is held at a fixed angle with constant pressure on the gas or brake pedal and matches the well-known bicycle model. Past works that use acceleration to compute TTC consider only longitudinally aligned acceleration. We additionally develop and present the Second-Order Time-to-Collision Algorithm using Region-based search (STAR) to efficiently compute the proposed second-order TTC and overcome the current...

So You Think You Can Track?

Gloudemans, Derek
Zachár, Gergely
Wang, Yanbing
Ji, Junyi
Nice, Matt
Bunting, Matt
Barbour, William
Sprinkle, Jonathan
Piccoli, Benedetto
Monache, Maria Laura Delle
Bayen, Alexandre
Seibold, Benjamin
Work, Daniel B.
2024

This work introduces a multi-camera tracking dataset consisting of 234 hours of video data recorded concurrently from 234 overlapping HD cameras covering a 4.2 mile stretch of 8-10 lane interstate highway near Nashville, TN. The video is recorded during a period of high traffic density with 500+ objects typically visible within the scene and typical object longevities of 3-15 minutes. GPS trajectories from 270 vehicle passes through the scene are manually corrected in the video data to provide a set of ground-truth trajectories for recall-oriented tracking metrics, and object...

Traffic Control via Connected and Automated Vehicles (CAVs): An Open-Road Field Experiment with 100 CAVs

Lee, Jonathan W.
Wang, Han
Jang, Kathy
Lichtle, Nathan
Hayat, Amaury
Bunting, Matthew
Alanqary, Arwa
Barbour, William
Fu, Zhe
Gong, Xiaoqian
Gunter, George
Hornstein, Sharon
Kreidieh, Abdul Rahman
Nice, Matthew W.
Richardson, William A.
Shah, Adit
Vinitsky, Eugene
Wu, Fangyu
Xiang, Shengquan
Almatrudi, Sulaiman
Althukair, Fahd
Bhadani, Rahul
Carpio, Joy
Chekroun, Raphael
Cheng, Eric
Chiri, Maria Teresa
Chou, Fang-Chieh
Delorenzo, Ryan
Gibson, Marsalis
Gloudemans, Derek
Gollakota, Anish
Ji, Junyi
Keimer, Alexander
Khoudari, Nour
Mahmood, Malaika
Mahmood, Mikail
Matin, Hossein Nick Zinat
McQuade, Sean
Ramadan, Rabie
Urieli, Daniel
Wang, Xia
Wang, Yanbing
Xu, Rita
Yao, Mengsha
You, Yiling
Zachár, Gergely
Zhao, Yibo
Ameli, Mostafa
Baig, Mirza Najamuddin
Bhaskaran, Sarah
Butts, Kenneth
Gowda, Manasi
Janssen, Caroline
Lee, John
Pedersen, Liam
Wagner, Riley
Zhang, Zimo
Zhou, Chang
Work, Daniel B.
Seibold, Benjamin
Sprinkle, Jonathan
Piccoli, Benedetto
Monache, Maria Laura Delle
Bayen, Alexandre M.
2025

The CIRCLES project aims to reduce instabilities in traffic flow, which are naturally occurring phenomena due to human driving behavior. Also called “phantom jams” or “stop-and-go waves,” these instabilities are a significant source of wasted energy. Toward this goal, the CIRCLES project designed a control system, referred to as the MegaController by the CIRCLES team, that could be deployed in real traffic. Our field experiment, the MegaVanderTest (MVT), leveraged a heterogeneous fleet of 100 longitudinally controlled vehicles as Lagrangian traffic actuators, each of which ran a controller...

Traffic Smoothing Using Explicit Local Controllers

Hayat, Amaury
Alanqary, Arwa
Bhadani, Rahul
Denaro, Christopher
Weightman, Ryan J.
Xiang, Shengquan
Lee, Jonathan W.
Bunting, Matthew
Gollakota, Anish
Nice, Matthew W.
Gloudemans, Derek
Zachár, Gergely
Davis, Jon F.
Monache, Maria Laura Delle
Seibold, Benjamin
Bayen, Alexandre M.
Sprinkle, Jonathan
Work, Daniel B.
Piccoli, Benedetto
2023

The dissipation of stop-and-go waves attracted recent attention as a traffic management problem, which can be efficiently addressed by automated driving. As part of the 100 automated vehicles experiment named MegaVanderTest, feedback controls were used to induce strong dissipation via velocity smoothing. More precisely, a single vehicle driving differently in one of the four lanes of I-24 in the Nashville area was able to regularize the velocity profile by reducing oscillations in time and velocity differences among vehicles. Quantitative measures of this effect were possible due to the...

Traffic Smoothing Using Explicit Local Controllers: Experimental Evidence for Dissipating Stop-and-go Waves with a Single Automated Vehicle in Dense Traffic

Hayat, Amaury
Alanqary, Arwa
Bhadani, Rahul
Denaro, Christopher
Weightman, Ryan J.
Xiang, Shengquan
Lee, Jonathan W.
Bunting, Matthew
Gollakota, Anish
Nice, Matthew W.
Gloudemans, Derek
Zachár, Gergely
Davis, Jon F.
Delle Monache, Maria Laura
Seibold, Benjamin
Bayen, Alexandre M.
Sprinkle, Jonathan
Work, Daniel B.
Piccoli, Benedetto
2025

This article presents experimental evidence of the ability of a single automated vehicle acting as a controller to effectively dissipate stop-and-go waves in real traffic. The automated vehicle succeeded in stabilizing the speed profile by reducing oscillations in time and speed variations between vehicles during rush hour on I-24 in the Nashville area. We detail the control design, deployment and results obtained in this experiment, conducted as part of the CIRCLES consortium’s “MegaVanderTest” 2022, which involved a total of 100 automated vehicles.

Urban Network Resilience Analysis and Equity Emphasized Recovery based on Reinforcement Learning

Wang, Han
Monache, Maria Laura Delle
2022

This paper introduces an equity emphasized re-covery planning method for urban traffic networks based on a data driven approach. An integrated evaluation index is proposed to assess equity in territorial accessibility during hazards recovery, which brings the variance in accessibility between communities as a penalty term into the overall accessibility. Taking the improvement of the integrated index as the reward function, the equity emphasized recovery control strategy is designed with a reinforcement learning algorithm to determine the recovery priority of the affected links. To test the...

A Discrete Choice Framework for Modeling and Forecasting the Adoption and Diffusion of New Transportation Services

El Zarwi, Feras
Vij, Akshay
Walker, Joan L.
2017

Major technological and infrastructural changes over the next decades, such as the introduction of autonomous vehicles, implementation of mileage-based fees, carsharing and ridesharing are expected to have a profound impact on lifestyles and travel behavior. Current travel demand models are unable to predict long-range trends in travel behavior as they do not entail a mechanism that projects membership and market share of new modes of transport (Uber, Lyft, etc.). We propose integrating discrete choice and technology adoption models to address the aforementioned issue. In order to do so,...

A Joint Model of Travel Information Acquisition and Response to Received Messages

Chorus, Caspar G.
Walker, Joan L.
Ben-Akiva, Moshe
2013

This paper presents a discrete-choice model of traveler response to information. It contributes to existing approaches by describing both the acquisition and the effect on travel choices of a variety of travel information types using a single integrative and parsimonious discrete-choice model. By doing so, the model captures the notion that both types of decisions (to acquire information and to execute a travel alternative) are the result of a single underlying system of preferences and beliefs. The model was estimated on choice sequences observed in a multimodal travel simulator...

A Revealed Preference Methodology to Evaluate Regret Minimization with Challenging Choice Sets: A Wildfire Evacuation Case Study

Wong, Stephen D.
Chorus, Caspar G.
Shaheen, Susan A.
Walker, Joan L.
2020

Regret is often experienced for difficult, important, and accountable choices. Consequently, we hypothesize that random regret minimization (RRM) may better describe evacuation behavior than traditional random utility maximization (RUM). However, in many travel related contexts, such as evacuation departure timing, specifying choice sets can be challenging due to unknown attribute levels and near-endless alternatives, for example. This has implications especially for estimating RRM models, which calculates attribute-level regret via pairwise comparison of attributes across all alternatives...