Traffic Operations and Management

Expert Level Control of Ramp Metering Based on Multi-Task Deep Reinforcement Learning

Belletti, Francois
Haziza, Daniel
Gomes, Gabriel
Bayen, Alexandre M.
2018

This paper shows how the recent breakthroughs in reinforcement learning (RL) that have enabled robots to learn to play arcade video games, walk, or assemble colored bricks, can be used to perform other tasks that are currently at the core of engineering cyberphysical systems. We present the first use of RL for the control of systems modeled by discretized non-linear partial differential equations (PDEs) and devise a novel algorithm to use non-parametric control techniques for large multi-agent systems. Cyberphysical systems (e.g., hydraulic channels, transportation systems, the energy grid...

Information Patterns in the Modeling and Design of Mobility Management Services

Keimer, Alexander
Laurent-Brouty, Nicolas
Farokhi, Farhad
Signargout, Hippolyte
Cvetkovic, Vladimir
Bayen, Alexandre M.
Johansson, Karl H.
2018

The development of sustainable transportation infrastructure for people and goods, using new technology and business models, can prove beneficial or detrimental for mobility, depending on its design and use. The focus of this paper is on the increasing impact new mobility services have on traffic patterns and transportation efficiency in general. Over the last decade, the rise of the mobile internet and the usage of mobile devices have enabled ubiquitous traffic information. With the increased adoption of specific smartphone applications, the number of users of routing applications has...

Stabilizing Traffic with Autonomous Vehicles

Wu, Cathy
Bayen, Alexandre M.
Mehta, Ankur
2018

Autonomous vehicles promise safer roads, energy savings, and more efficient use of existing infrastructure, among many other benefits. Although the effect of autonomous vehicles has been studied in the limits (near-zero or full penetration), the transition range requires new formulations, mathematical modeling, and control analysis. In this article, we study the ability of small numbers of autonomous vehicles to stabilize a single-lane system of human-driven vehicles. We formalize the problem in terms of linear string stability, derive optimality conditions from frequency-domain analysis,...

Stability and Implementation of a Cycle-Based Max Pressure Controller for Signalized Traffic Networks

Anderson, Leah
Pumir, Thomas
Triantafyllos, Dimitrios
Bayen, Alexandre M.
2018

Intelligent use of network capacity via responsive signal control will become increasingly essential as congestion increases on urban roadways. Existing adaptive control systems require lengthy location-specific tuning procedures or expensive central communications infrastructure. Previous theoretical work proposed the application of a max pressure controller to maximize network throughput in a distributed manner with minimal calibration. Yet this algorithm as originally formulated has unpractical hardware and safety constraints. We fundamentally alter the formulation of the max pressure...

Discrete-Time System Optimal Dynamic Traffic Assignment (SO-DTA) with Partial Control for Physical Queuing Networks

Samaranayake, Samitha
Krichene, Walid
Reilly, Jack
Monache, Maria Laura Delle
Goatin, Paola
Bayen, Alexandre
2018

We consider the System Optimal Dynamic Traffic Assignment (SO-DTA) problem with Partial Control for general networks with physical queuing. Our goal is to optimally control any subset of the networks agents to minimize the total congestion of all agents in the network. We adopt a flow dynamics model that is a Godunov discretization of the Lighthill–Williams–Richards partial differential equation with a triangular flux function and a corresponding multicommodity junction solver. The partial control formulation generalizes the SO-DTA problem to consider cases where only a fraction of the...

Privacy-preserving MaaS fleet management

Belletti, Francois
Bayen, Alexandre M.
2018

On-demand traffic fleet optimization requires operating Mobility as a Service (MaaS) companies such as Uber, Lyft to locally match the offer of available vehicles with their expected number of requests referred to as demand (as well as to take into account other constraints such as driver’s schedules and preferences). In the present article, we show that this problem can be encoded into a Constrained Integer Quadratic Program (CIQP) with block independent constraints that can then be relaxed in the form of a convex optimization program. We leverage this particular structure to yield a...

Benchmarks for Reinforcement Learning in Mixed-Autonomy Traffic

Vinitsky, Eugene
Kreidieh, Aboudy
Flem, Luc Le
Kheterpal, Nishant
Jang, Kathy
Bayen, Alexandre M.
2018

We release new benchmarks in the use of deep reinforcement learning (RL) to create controllers for mixed-autonomy traffic, where connected and autonomous vehicles (CAVs) interact with human drivers and infrastructure. Benchmarks, such as Mujoco or the Arcade Learning Environment, have spurred new research by enabling researchers to effectively compare their results so that they can focus on algorithmic improvements and control techniques rather than system design. To promote similar advances in traffic control via RL, we propose four benchmarks, based on three new traffic scenarios,...

Measuring Regret in Routing: Assessing the Impact of Increased App Usage

Cabannes, Theophile
Shyu, Frank
Porter, Emily
Yao, Shuai
Wang, Yexin
2018

This article is focused on measuring the impact of navigational apps on road traffic patterns. We first define the marginal regret, which characterizes the difference between the travel time experienced on the most optimal path and the path of interest between the same origin destination pair. We then introduce a new metric, the average marginal regret, which is the average of marginal regret, taken over all possible OD pairs in the network. We evaluate the average marginal regret in simulations with varying proportions of app and non-app users (information vs. no information) using the...

Structural Analysis of Specific Environmental Traffic Assignment Problems

Khiyami, Aziz
Keimer, Alexander
Bayen, Alexandre
2018

The goal of this article is to develop a framework for Environmental Traffic Assignment (E-TAP); that is a methodology for allocating traffic flows on a road network with the objective of minimizing objective functions related to energy such as fuel consumption or traffic pollutants. We investigate the underlying minimization problem in E-TAP which we characterize and study for uniqueness. This study is accomplished by exploiting convexity properties of the developed environmental objective functions and obtaining parameter sets for which the objective functions are strictly convex. The...

Dissipating Stop-and-Go Waves in Closed and Open Networks via Deep Reinforcement Learning

Kreidieh, Abdul Rahman
Wu, Cathy
Bayen, Alexandre M.
2018

This article demonstrates the ability for model-free reinforcement learning (RL) techniques to generate traffic control strategies for connected and automated vehicles (CAVs) in various network geometries. This method is demonstrated to achieve near complete wave dissipation in a straight open road network with only 10% CAV penetration, while penetration rates as low as 2.5% are revealed to contribute greatly to reductions in the frequency and magnitude of formed waves. Moreover, a study of controllers generated in closed network scenarios exhibiting otherwise similar densities and...