Intelligent Transportation Systems

Expert Level Control of Ramp Metering Based on Multi-Task Deep Reinforcement Learning

Belletti, Francois
Haziza, Daniel
Gomes, Gabriel
Bayen, Alexandre M.
2018

This paper shows how the recent breakthroughs in reinforcement learning (RL) that have enabled robots to learn to play arcade video games, walk, or assemble colored bricks, can be used to perform other tasks that are currently at the core of engineering cyberphysical systems. We present the first use of RL for the control of systems modeled by discretized non-linear partial differential equations (PDEs) and devise a novel algorithm to use non-parametric control techniques for large multi-agent systems. Cyberphysical systems (e.g., hydraulic channels, transportation systems, the energy grid...

Information Patterns in the Modeling and Design of Mobility Management Services

Keimer, Alexander
Laurent-Brouty, Nicolas
Farokhi, Farhad
Signargout, Hippolyte
Cvetkovic, Vladimir
Bayen, Alexandre M.
Johansson, Karl H.
2018

The development of sustainable transportation infrastructure for people and goods, using new technology and business models, can prove beneficial or detrimental for mobility, depending on its design and use. The focus of this paper is on the increasing impact new mobility services have on traffic patterns and transportation efficiency in general. Over the last decade, the rise of the mobile internet and the usage of mobile devices have enabled ubiquitous traffic information. With the increased adoption of specific smartphone applications, the number of users of routing applications has...

Measuring Regret in Routing: Assessing the Impact of Increased App Usage

Cabannes, Theophile
Shyu, Frank
Porter, Emily
Yao, Shuai
Wang, Yexin
2018

This article is focused on measuring the impact of navigational apps on road traffic patterns. We first define the marginal regret, which characterizes the difference between the travel time experienced on the most optimal path and the path of interest between the same origin destination pair. We then introduce a new metric, the average marginal regret, which is the average of marginal regret, taken over all possible OD pairs in the network. We evaluate the average marginal regret in simulations with varying proportions of app and non-app users (information vs. no information) using the...

Integrated Offline and Online Optimization-Based Control in a Base-Parallel Architecture

Jamshidnejad, Anahita
Gomes, Gabriel
Bayen, Alexandre M.
Schutter, Bart De
2019

We propose an integrated control architecture to address the gap that currently exists for efficient real-time implementation of MPC-based control approaches for highly nonlinear systems with fast dynamics and a large number of control constraints. The proposed architecture contains two types of controllers: base controllers that are tuned or optimized offline, and parallel controllers that solve an optimization-based control problem online. The control inputs computed by the base controllers provide starting points for the optimization problem of the parallel controllers, which operate in...

Regrets in Routing Networks: Measuring the Impact of Routing Apps in Traffic

Cabannes, Theophile
Sangiovanni, Marco
Keimer, Alexander
Bayen, Alexandre M.
2019

The impact of the recent increase in routing apps usage on road traffic remains uncertain to this day. The article introduces, for the first time, a criterion to evaluate a distance between an observed state of traffic and the user equilibrium of the traffic assignment: the average marginal regret. The average marginal regret provides a quantitative measure of the impact of routing apps on traffic using only link flows, link travel times, and travel demand. In non-atomic routing games (or static traffic assignment models), the average marginal regret is a measure of selfish drivers’...

Deep Truck : A Deep Neural Network Model for Longitudinal Dynamics of Heavy Duty Trucks

Albeaik, Saleh
Chou, Fang-Chieh
Lu, Xiao-Yun
Bayen, Alexandre M.
2019

This article demonstrates the use of deep neural networks (NN) and deep reinforcement learning (deep-RL) for modeling and control of longitudinal heavy duty truck dynamics. Instead of explicit use of analytical model derived information or parameters about the truck, the deep NN model is fitted to data using a brief set of historical data collected from an arbitrary driving cycle. The deep model is used in this article to design a cruise controller for the truck using model-free deep-RL. The deep model and the control loop performances are demonstrated both using state-of-the-art...

Guardians of the Deep Fog: Failure-Resilient DNN Inference from Edge to Cloud

Yousefpour, Ashkan
Devic, Siddartha
Nguyen, Brian Q.
Kreidieh, Aboudy
Liao, Alan
Bayen, Alexandre M.
Jue, Jason P.
2019

Partitioning and distributing deep neural networks (DNNs) over physical nodes such as edge, fog, or cloud nodes, could enhance sensor fusion, and reduce bandwidth and inference latency. However, when a DNN is distributed over physical nodes, failure of the physical nodes causes the failure of the DNN units that are placed on these nodes. The performance of the inference task will be unpredictable, and most likely, poor, if the distributed DNN is not specifically designed and properly trained for failures. Motivated by this, we introduce deepFogGuard, a DNN architecture augmentation scheme...

Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design

Dennis, Michael
Jaques, Natasha
Vinitsky, Eugene
Bayen, Alexandre
Russell, Stuart
Critch, Andrew
Levine, Sergey
2020

A wide range of reinforcement learning (RL) problems --- including robustness, transfer learning, unsupervised RL, and emergent complexity --- require specifying a distribution of tasks or environments in which a policy will be trained. However, creating a useful distribution of environments is error prone, and takes a significant amount of developer time and effort. We propose Unsupervised Environment Design (UED) as an alternative paradigm, where developers provide environments with unknown parameters, and these parameters are used to automatically produce a distribution over valid,...

Routing on Traffic Networks Incorporating Past Memory up to Real-Time Information on the Network State

Keimer, Alexander
Bayen, Alexandre
2020

In this review, we discuss routing algorithms for the dynamic traffic assignment (DTA) problem that assigns traffic flow in a given road network as realistically as possible. We present a new class of so-called routing operators that route traffic flow at intersections based on either real-time information about the status of the network or historical data. These routing operators thus cover the distribution of traffic flow at all possible intersections. To model traffic flow on the links, we use a well-known macroscopic ordinary delay differential equation. We prove the existence and...

BISTRO: Berkeley Integrated System for Transportation Optimization

Feygin, Sidney A.
Lazarus, Jessica
Forscher, Edward H.
Golfier-Vetterli, Valentine
Lee, Jonathan W.
Gupta, Abhishek
Bayen, Alexandre
2020

The current trend toward urbanization and adoption of flexible and innovative mobility technologies will have complex and difficult-to-predict effects on urban transportation systems. Comprehensive methodological frameworks that account for the increasingly uncertain future state of the urban mobility landscape do not yet exist. Furthermore, few approaches have enabled the massive ingestion of urban data in planning tools capable of offering the flexibility of scenario-based design.This article introduces Berkeley Integrated System for Transportation Optimization (BISTRO), a new open...