ITS Berkeley

Framework for Control and Deep Reinforcement Learning in Traffic

Wu, Cathy
Parvate, Kanaad
Kheterpal, Nishant
Dickstein, Leah
Mehta, Ankur
Vinitsky, Eugene
Bayen, Alexandre M.
2017

Recent advances in deep reinforcement learning (RL) offer an opportunity to revisit complex traffic control problems at the level of vehicle dynamics, with the aim of learning locally optimal policies (with respect to the policy parameterization) for a variety of objectives such as matching a target velocity or minimizing fuel consumption. In this article, we present a framework called CISTAR (Customized Interface for SUMO, TraCI, and RLLab) that integrates the widely used traffic simulator SUMO with a standard deep reinforcement learning library RLLab. We create an interface allowing for...

Multi-Lane Reduction: A Stochastic Single-Lane Model for Lane Changing

Wu, Cathy
Vinitsky, Eugene
Kreidieh, Aboudy
Bayen, Alexandre
2017

Lane changes can induce natural large perturbations in traffic flow and are known to impact traffic throughput and energy consumption. Their precise effects are understudied. The primary aim of this article is to present a model for lane changing that is tractable for system-level analysis and yet captures qualities of microscopic vehicle dynamics. We present a stochastic lane changing model, which permits a two-stage reduction: 1) of the (microscopic) multi-lane problem into a stochastic single-lane problem, and 2) of the stochastic single-lane model into a Markov chain macroscopic model...

Reduction in Fall Rate in Dementia Managed Care Through Video Incident Review: Pilot Study

Bayen, Eleonore
Jacquemot, Julien
Netscher, George
Agrawal, Pulkit
Noyce, Lynn Tabb
Bayen, Alexandre
2017

Background: Falls of individuals with dementia are frequent, dangerous, and costly. Early detection and access to the history of a fall is crucial for efficient care and secondary prevention in cognitively impaired individuals. However, most falls remain unwitnessed events. Furthermore, understanding why and how a fall occurred is a challenge. Video capture and secure transmission of real-world falls thus stands as a promising assistive tool. Objective: The objective of this study was to analyze how continuous video monitoring and review of falls of individuals with dementia can support...

Emergent Behaviors in Mixed-Autonomy Traffic

Wu, Cathy
Kreidieh, Aboudy
Vinitsky, Eugene
Bayen, Alexandre M.
2017

Traffic dynamics are often modeled by complex dynamical systems for which classical analysis tools can struggle to provide tractable policies used by transportation agencies and planners. In light of the introduction of automated vehicles into transportation systems, there is a new need for understanding the impacts of automation on transportation networks. The present article formulates and approaches the mixed-autonomy traffic control problem (where both automated and human-driven vehicles are present) using the powerful framework of deep reinforcement learning (RL). The resulting...

Estimation of Performance Metrics at Signalized Intersections Using Loop Detector Data and Probe Travel Times

Gan, Qijian
Gomes, Gabriel
Bayen, Alexandre
2017

This paper introduces a simple but practical approach that uses both loop detector data and probe travel times for computing the vehicle hours traveled (VHT), average delay, and level of service (LOS) for signalized intersections. The goal is to improve upon the state-of-the-practice method outlined in the highway capacity manual (HCM) by incorporating additional travel time measurements from probe vehicles or vehicle re-identification systems. The proposed methodology is designed to work under a variety of traffic conditions, including states of congestion in which the HCM methodology is...

Learnability of Edge Cost Functions in Routing Games

Thai, Jérôme
Bayen, Alexandre M.
2017

We study the learnability of the edge cost functions in routing games from observations of the equilibrium flows, where the learnability is measured as the minimum number of samples needed to maintain a high consistency of the empirical risk, which is a statistical estimator for the quality of the learned model. To provide an upper bound on the minimum sample size, we motivate the analysis of the uniform laws of large numbers on a class of loss functions indexed over the space of parameters we want to estimate. On one hand, leveraging results on the complexity of function classes and in...

Preserving Privacy in Road User Charge (RUC) Architectures

Bayen, Alexandre
Forscher, Teddy
Shaheen, Susan
2018

One of the major concerns for the technical implementation of a RUC is the ability to collect the mileage of motorists in a way that preserves and protects individual privacy. With the widespread use of connected devices/smartphones and the growth of connected vehicles and the existence of toll tag readers, it is possible to build and deploy architectures capable of computing advanced fee structures (based upon on mileage, road type, time of day, and speed, among other features) that respect motorist privacy. A possible architecture can rely on the use of virtual trip lines (VTLs) –a...

The Impact of GPS-Enabled Shortest Path Routing on Mobility: A Game Theoretic Approach

Cabannes, Theophile
Vincentelli, Marco Antonio Sangiovanni
Sundt, Alexander
Signargout, Hippolyte
Porter, Emily
Bayen, Alexandre
2018

This article investigates the impact of app use on traffic patterns. With ubiquity of traffic information and the increased use of routing apps, urban and suburban areas in the US have seen a recent rise in “cut-through” traffic and related congestion patterns. This increase is suspected to be both an instantaneous phenomenon (a natural response of routing apps to special events, accidents, or other problems reducing capacity locally in transportation networks) and a trend (progressive increase of such traffic over time, with a corresponding shift in demand on the transportation...

Road Usage Charging (RUC)

Forscher, Teddy
Bayen, Alexandre
Shaheen, Susan
2018

Pricing transportation infrastructure, either to achieve a desired outcome or to raise revenue, is a concept dating back to early-and mid-20thcentury economics and transportation scholarship. Different approaches to pricing (e.g., area-wide pricing, vehicle miles traveled, express lanes, etc.) have been adopted in parts of Europe and Asia; some strategies cover all road users, some only passenger vehicles, and others only commercial and goods movement vehicles. Pricing, as a revenue source, has recently gained momentum in the U.S., driven by federal legislation (MAP-21; FAST Act) and state...

On Learning How Players Learn: Estimation of Learning Dynamics in the Routing Game

Krichene, Walid
Bourguiba, Mohamed Chedhli
Tlam, Kiet
Bayen, Alexandre
2018

The routing game models congestion in transportation networks, communication networks, and other cyber-physical systems in which agents compete for shared resources. We consider an online learning model of player dynamics: at each iteration, every player chooses a route (or a probability distribution over routes, which corresponds to a flow allocation over the physical network), then the joint decision of all players determines the costs of each path, which are then revealed to the players.We pose the following estimation problem: given a sequence of player decisions and the corresponding...