ITS Berkeley

Deep Truck Cruise Control: Field Experiments and Validation of Heavy Duty Truck Cruise Control Using Deep Reinforcement Learning

Albeaik, Saleh
Wu, Trevor
Vurimi, Ganeshnikhil
Chou, Fang-Chieh
Lu, Xiao-Yun
Bayen, Alexandre M.
2022

Building control systems for heavy duty trucks have historically been dependent on availability of the details of the mechanical configuration of each target truck. This article investigates transfer and robustness of continuous control systems learned using model free deep-RL as an alternative; a configuration agnostic strategy for control system development. For this purpose, deep-RL cruise control policies are developed and validated in simulation and field experiments using two differently configured trucks; full-size Volvo and Freightliner trucks. Their performance are validated for...

Using Automated Vehicle (AV) Technology to Smooth Traffic Flow and Reduce Greenhouse Gas Emissions

Almatrudi, Sulaiman
Parvate, Kanaad
Rothchild, Daniel
Vijay, Upadhi
2022

Passenger and heavy-duty vehicles make up 36% of California’s greenhouse gas (GHG) emissions. Reducing emissions from vehicular travel is therefore paramount for any path towards carbon neutrality. Efforts to reduce GHGs by encouraging mode shift or increasing vehicle efficiency are, and will continue to be, a critical part of decarbonizing the transportation sector. Emerging technologies are creating an opportunity to reduce GHGs. Human driving behaviors in congested traffic have been shown to create stop-and-go waves. When waves form, cars periodically slow down (sometimes to a stop) and...

Flow: A Modular Learning Framework for Mixed Autonomy Traffic

Wu, Cathy
Kreidieh, Abdul Rahman
Parvate, Kanaad
Vinitsky, Eugene
Bayen, Alexandre M.
2022

The rapid development of autonomous vehicles (AVs) holds vast potential for transportation systems through improved safety, efficiency, and access to mobility. However, the progression of these impacts, as AVs are adopted, is not well understood. Numerous technical challenges arise from the goal of analyzing the partial adoption of autonomy: partial control and observation, multivehicle interactions, and the sheer variety of scenarios represented by real-world networks. To shed light into near-term AV impacts, this article studies the suitability of deep reinforcement learning (RL) for...

Deep Truck Cruise Control: Field Experiments and Validation of Heavy Duty Truck Cruise Control Using Deep Reinforcement Learning

Albeaik, Saleh
Wu, Trevor
Vurimi, Ganeshnikhil
Chou, Fang-Chieh
Bayen, Alexandre M.
2022

Building control systems for heavy duty trucks have historically been dependent on availability of the details of the mechanical configuration of each target truck. This article investigates transfer and robustness of continuous control systems learned using model free deep-RL as an alternative; a configuration agnostic strategy for control system development. For this purpose, deep-RL cruise control policies are developed and validated in simulation and field experiments using two differently configured trucks; full-size Volvo and Freightliner trucks. Their performance are validated for...

Using Automated Vehicle (AV) Technology to Smooth Traffic Flow and Reduce Greenhouse Gas Emissions

Almatrudi, Sulaiman
Parvate, Kanaad
Rothchild, Daniel
Vijay, Upadhi
2022

Passenger and heavy-duty vehicles make up 36% of California’s greenhouse gas (GHG) emissions. Reducing emissions from vehicular travel is therefore paramount for any path towards carbon neutrality. Efforts to reduce GHGs by encouraging mode shift or increasing vehicle efficiency are, and will continue to be, a critical part of decarbonizing the transportation sector. Emerging technologies are creating an opportunity to reduce GHGs. Human driving behaviors in congested traffic have been shown to create stop-and-go waves. When waves form, cars periodically slow down (sometimes to a stop) and...

Deploying Traffic Smoothing Cruise Controllers Learned from Trajectory Data

Lichtle, Nathan
Vinitsky, Eugene
Nice, Matthew
Seibold, Benjamin
Work, Dan
Bayen, Alexandre M.
2022

Autonomous vehicle-based traffic smoothing con-trollers are often not transferred to real-world use due to challenges in calibrating many-agent traffic simulators. We show a pipeline to sidestep such calibration issues by collecting trajectory data and learning controllers directly from trajectory data that are then deployed zero-shot onto the highway. We construct a dataset of 772.3 kilometers of recorded drives on the I–24. We then construct a simple simulator using the recorded drives as the lead vehicle in front of a simulated platoon consisting of one autonomous vehicle and five human...

Learning Energy-Efficient Driving Behaviors by Imitating Experts

Rahman Kreidieh, Abdul
Fu, Zhe
Bayen, Alexandre M.
2022

The rise of vehicle automation has generated significant interest in the potential role of future automated vehicles (AVs). In particular, in highly dense traffic settings, AVs are expected to serve as congestion-dampeners, mitigating the presence of instabilities that arise from various sources. However, in many applications, such maneuvers rely heavily on non-local sensing or coordination by interacting AVs, thereby rendering their adaptation to real-world settings a particularly difficult challenge. To address this challenge, this paper examines the role of imitation learning in...

Modeling Multilane Traffic with Moving Obstacles by Nonlocal Balance Laws

Bayen, Alexandre
Friedrich, Jan
Keimer, Alexander
Pflug, Lukas
Veeravalli, Tanya
2022

We introduce a Follow-the-Leader approximation of a nonlocal generalized Aw--Rascle--Zhang (GARZ) model for traffic flow. We prove the convergence to weak solutions of the corresponding macroscopic equations deriving $L^\infty$ and BV estimates. We also provide numerical simulations illustrating the micro-macro convergence and we numerically investigate the nonlocal to local limit for both the microscopic and macroscopic models.

A Rigorous Multi-Population Multi-Lane Hybrid Traffic Model for Dissipation of Waves via Autonomous Vehicles

Kardous, Nicolas
Hayat, Amaury
McQuade, Sean T.
Gong, Xiaoqian
Truong, Sydney
Bayen, Alexandre
2022

In this paper, a multi-lane multi-population microscopic model, which presents stop-and-go waves, is proposed to simulate traffic on a ring-road. Vehicles are divided between human-driven and autonomous vehicles (AV). Control strategies are designed with the ultimate goal of using a small number of AVs (less than 5% penetration rate) to represent Lagrangian control actuators that can smooth the multilane traffic flow and dissipate the traffic instabilities, and in particular stop-and-go waves. This in turn may reduce fuel consumption and emissions. The lane-changing mechanism is based on...

Reinforcement Learning Versus PDE Backstepping and PI Control for Congested Freeway Traffic

Yu, Huan
Park, Saehong
Bayen, Alexandre
2022

We develop reinforcement learning (RL) boundary controllers to mitigate stop-and-go traffic congestion on a freeway segment. The traffic dynamics of the freeway segment are governed by a macroscopic Aw–Rascle–Zhang (ARZ) model, consisting of 2 \times 2 quasi-linear partial differential equations (PDEs) for traffic density and velocity. The boundary stabilization of the linearized ARZ PDE model has been solved by PDE backstepping, guaranteeing spatial L<sup>2</sup> norm regulation of the traffic state to uniform density and velocity and ensuring that traffic oscillations are...