Traffic Operations and Management

Expert Level Control of Ramp Metering Based on Multi-Task Deep Reinforcement Learning

Belletti, Francois
Haziza, Daniel
Gomes, Gabriel
Alexandre Bayen
2018

This paper shows how the recent breakthroughs in reinforcement learning (RL) that have enabled robots to learn to play arcade video games, walk, or assemble colored bricks, can be used to perform other tasks that are currently at the core of engineering cyberphysical systems. We present the first use of RL for the control of systems modeled by discretized non-linear partial differential equations (PDEs) and devise a novel algorithm to use non-parametric control techniques for large multi-agent systems. Cyberphysical systems (e.g., hydraulic channels, transportation systems, the energy grid...

Information Patterns in the Modeling and Design of Mobility Management Services

Keimer, Alexander
Laurent-Brouty, Nicolas
Farokhi, Farhad
Signargout, Hippolyte
Cvetkovic, Vladimir
Alexandre Bayen
Johansson, Karl H.
2018

The development of sustainable transportation infrastructure for people and goods, using new technology and business models, can prove beneficial or detrimental for mobility, depending on its design and use. The focus of this paper is on the increasing impact new mobility services have on traffic patterns and transportation efficiency in general. Over the last decade, the rise of the mobile internet and the usage of mobile devices have enabled ubiquitous traffic information. With the increased adoption of specific smartphone applications, the number of users of routing applications has...

Stabilizing Traffic with Autonomous Vehicles

Wu, Cathy
Alexandre Bayen
Mehta, Ankur
2018

Autonomous vehicles promise safer roads, energy savings, and more efficient use of existing infrastructure, among many other benefits. Although the effect of autonomous vehicles has been studied in the limits (near-zero or full penetration), the transition range requires new formulations, mathematical modeling, and control analysis. In this article, we study the ability of small numbers of autonomous vehicles to stabilize a single-lane system of human-driven vehicles. We formalize the problem in terms of linear string stability, derive optimality conditions from frequency-domain analysis,...

Stability and Implementation of a Cycle-Based Max Pressure Controller for Signalized Traffic Networks

Anderson, Leah
Pumir, Thomas
Triantafyllos, Dimitrios
Alexandre Bayen
2018

Intelligent use of network capacity via responsive signal control will become increasingly essential as congestion increases on urban roadways. Existing adaptive control systems require lengthy location-specific tuning procedures or expensive central communications infrastructure. Previous theoretical work proposed the application of a max pressure controller to maximize network throughput in a distributed manner with minimal calibration. Yet this algorithm as originally formulated has unpractical hardware and safety constraints. We fundamentally alter the formulation of the max pressure...

A Decision Support System for Evaluating the Impacts of Routing Applications on Urban Mobility

Lazarus, Jessica
Ugirumurera, Juliette
Hinardi, Stefanus
Zhao, Michael
Shyu, Frank
Alexandre Bayen
2018

The rise of congestion across the United States and the increasing adoption of mobile routing services have enabled drivers with the ability to find the fastest routes available in urban road networks. Arterial roads and side streets originally designed for local traffic are impacted by the influx of selfishly routed drivers, garnering much recent media attention and civic debate. Classic flow-based game theoretic models provide the framework for simulating the behavior of routed and non-routed drivers on a road network. We developed an interactive policy decision support system called the...

Discrete-Time System Optimal Dynamic Traffic Assignment (SO-DTA) with Partial Control for Physical Queuing Networks

Samaranayake, Samitha
Krichene, Walid
Reilly, Jack
Monache, Maria Laura Delle
Goatin, Paola
Alexandre Bayen
2018

We consider the System Optimal Dynamic Traffic Assignment (SO-DTA) problem with Partial Control for general networks with physical queuing. Our goal is to optimally control any subset of the networks agents to minimize the total congestion of all agents in the network. We adopt a flow dynamics model that is a Godunov discretization of the Lighthill–Williams–Richards partial differential equation with a triangular flux function and a corresponding multicommodity junction solver. The partial control formulation generalizes the SO-DTA problem to consider cases where only a fraction of the...

Benchmarks for Reinforcement Learning in Mixed-Autonomy Traffic

Vinitsky, Eugene
Kreidieh, Aboudy
Flem, Luc Le
Kheterpal, Nishant
Jang, Kathy
Alexandre Bayen
2018

We release new benchmarks in the use of deep reinforcement learning (RL) to create controllers for mixed-autonomy traffic, where connected and autonomous vehicles (CAVs) interact with human drivers and infrastructure. Benchmarks, such as Mujoco or the Arcade Learning Environment, have spurred new research by enabling researchers to effectively compare their results so that they can focus on algorithmic improvements and control techniques rather than system design. To promote similar advances in traffic control via RL, we propose four benchmarks, based on three new traffic scenarios,...

Structural Analysis of Specific Environmental Traffic Assignment Problems

Khiyami, Aziz
Keimer, Alexander
Alexandre Bayen
2018

The goal of this article is to develop a framework for Environmental Traffic Assignment (E-TAP); that is a methodology for allocating traffic flows on a road network with the objective of minimizing objective functions related to energy such as fuel consumption or traffic pollutants. We investigate the underlying minimization problem in E-TAP which we characterize and study for uniqueness. This study is accomplished by exploiting convexity properties of the developed environmental objective functions and obtaining parameter sets for which the objective functions are strictly convex. The...

Lagrangian Control through Deep-RL: Applications to Bottleneck Decongestion

Vinitsky, Eugene
Parvate, Kanaad
Kreidieh, Aboudy
Wu, Cathy
Alexandre Bayen
2018

Using deep reinforcement learning, we derive novel control policies for autonomous vehicles to improve the throughput of a bottleneck modeled after the San Francisco-Oakland Bay Bridge. Using Flow, a new library for applying deep reinforcement learning to traffic micro-simulators, we consider the problem of improving the throughput of a traffic benchmark: a two-stage bottleneck where four lanes reduce to two and then reduce to one. We first characterize the inflow-outflow curve of this bottleneck without any control. We introduce an inflow of autonomous vehicles with the intent of...

A Unified Software Framework to Enable Solution of Traffic Assignment Problems at Extreme Scale

Ugirumurera, Juliette
Gomes, Gabriel
Porter, Emily
Li, Xiaoye S.
Alexandre Bayen
2018

We describe a modular software framework for solving user equilibrium traffic assignment problems. The design is based on the formulation of the problem as a variational inequality. Unlike most existing traffic assignment software which focus on specific traffic models, our framework accommodates various traffic models, but also enables using parallel computation in high performance computing environments to speed up large-scale equilibrium calculations. We compare the solutions obtained under several models: static, Merchant-Nemhauser, `CTM with instantaneous travel time', and `CTM with...