ITS Berkeley

Imputing a Variational Inequality Function or a Convex Objective Function: A Robust Approach

Thai, Jérôme
Bayen, Alexandre M.
2018

To impute the function of a variational inequality and the objective of a convex optimization problem from observations of (nearly) optimal decisions, previous approaches constructed inverse programming methods based on solving a convex optimization problem [17], [7]. However, we show that, in addition to requiring complete observations, these approaches are not robust to measurement errors, while in many applications, the outputs of decision processes are noisy and only partially observable from, e.g., limitations in the sensing infrastructure. To deal with noisy and missing data, we...

On Social Optimal Routing Under Selfish Learning

Krichene, Walid
Suarez Castillo, Milena
Bayen, Alexandre M.
2018

We consider a repeated routing game over a finite horizon with partial control under selfish response, in which a central authority can control a fraction of the flow and seeks to improve a network-wide objective, while the remaining flow applies an online learning algorithm. This finite horizon control problem is inspired from the one-shot Stackelberg routing game. Our setting is different in that we do not assume that the selfish players play a Nash equilibrium; instead, we assume that they apply an online learning algorithm. This results in an optimal control problem under learning...

Resiliency of Mobility-as-a-Service Systems to Denial-of-Service Attacks

Thai, Jérôme
Yuan, Chenyang
Bayen, Alexandre M.
2018

Mobility-as-a-Service (MaaS) systems, such as ride-sharing services, have expanded very quickly over the past years. However, the popularity of MaaS systems make them increasingly vulnerable to denial-of-service (DOS) attacks, in which attackers attempt to disrupt the system to make it unavailable to the customers. Expanding on an established queuing-theoretical model for MaaS systems, attacks are modeled as a malicious control of a fraction of vehicles in the network. We then formulate a stochastic control problem that maximizes the passenger loss in the network in steady state, and solve...

Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselines

Wu, Cathy
Rajeswaran, Aravind
Duan, Yan
Kumar, Vikash
Bayen, Alexandre M.
2018

Policy gradient methods have enjoyed great success in deep reinforcement learning but suffer from high variance of gradient estimates. The high variance problem is particularly exasperated in problems with long horizons or high-dimensional action spaces. To mitigate this issue, we derive a bias-free action-dependent baseline for variance reduction which fully exploits the structural form of the stochastic policy itself and does not make any additional assumptions about the MDP. We demonstrate and quantify the benefit of the action-dependent baseline through both theoretical analysis as...

Expert Level Control of Ramp Metering Based on Multi-Task Deep Reinforcement Learning

Belletti, Francois
Haziza, Daniel
Gomes, Gabriel
Bayen, Alexandre M.
2018

This paper shows how the recent breakthroughs in reinforcement learning (RL) that have enabled robots to learn to play arcade video games, walk, or assemble colored bricks, can be used to perform other tasks that are currently at the core of engineering cyberphysical systems. We present the first use of RL for the control of systems modeled by discretized non-linear partial differential equations (PDEs) and devise a novel algorithm to use non-parametric control techniques for large multi-agent systems. Cyberphysical systems (e.g., hydraulic channels, transportation systems, the energy grid...

Occupancy Detection via Environmental Sensing

Jin, Ming
Bekiaris-Liberis, Nikolaos
Weekly, Kevin
Spanos, Costas
Bayen, Alexandre M.
2018

Sensing by proxy (SbP) is proposed in this paper as a sensing paradigm for occupancy detection, where the inference is based on “proxy” measurements such as temperature and CO2 concentrations. The effects of occupants on indoor environments are captured by constitutive models comprising a coupled partial differential equation–ordinary differential equation system that exploits the spatial and physical features. Sensor fusion of multiple environmental parameters is enabled in the proposed framework. We report on experiments conducted under simulated conditions and real-life circumstances,...

Information Patterns in the Modeling and Design of Mobility Management Services

Keimer, Alexander
Laurent-Brouty, Nicolas
Farokhi, Farhad
Signargout, Hippolyte
Cvetkovic, Vladimir
Bayen, Alexandre M.
Johansson, Karl H.
2018

The development of sustainable transportation infrastructure for people and goods, using new technology and business models, can prove beneficial or detrimental for mobility, depending on its design and use. The focus of this paper is on the increasing impact new mobility services have on traffic patterns and transportation efficiency in general. Over the last decade, the rise of the mobile internet and the usage of mobile devices have enabled ubiquitous traffic information. With the increased adoption of specific smartphone applications, the number of users of routing applications has...

Extended, Continuous Measures of Functional Status in Community Dwelling Persons with Alzheimer’s and Related Dementia: Infrastructure, Performance, Tradeoffs, Preliminary Data, and Promise

Zylstra, Bradley
Netscher, George
Jacquemot, Julien
Schaffer, Michael
Shen, Galen
Bayen, Alexandre M.
2018

Background
The past decades have seen phenomenal growth in the availability of inexpensive and powerful personal computing devices. Efforts to leverage these devices to improve health care outcomes promise to remake many aspects of healthcare delivery, but remain in their infancy.
New method
We describe the development of a mobile health platform designed for daily measures of functional status in ambulatory, community dwelling subjects, including those who have Alzheimer’s disease or related neurodegenerative disorders. Using Smartwatches and Smartphones we measure subject...

A Unified Software Framework for Solving Traffic Assignment Problems

Ugirumurera, Juliette
Gomes, Gabriel
Porter, Emily
Li, Xiaoye S.
Bayen, Alexandre
2018

We describe a software framework for solving user equilibrium traffic assignment problems. The design is based on the formulation of the problem as a variational inequality. The software implements these as well as several numerical methods for find equilirbria. We compare the solutions obtained under several models: static, Merchant-Nemhauser, `CTM with instantaneous travel time', and `CTM with actual travel time'. Some important differences are demonstrated.

Building-in-Briefcase: A Rapidly-Deployable Environmental Sensor Suite for the Smart Building

Weekly, Kevin
Jin, Ming
Zou, Han
Hsu, Christopher
Soyza, Chris
Bayen, Alexandre M.
Spanos, Costas
2018

A building’s environment has profound influence on occupant comfort and health. Continuous monitoring of building occupancy and environment is essential to fault detection, intelligent control, and building commissioning. Though many solutions for environmental measuring based on wireless sensor networks exist, they are not easily accessible to households and building owners who may lack time or technical expertise needed to set up a system and get quick and detailed overview of environmental conditions. Building-in-Briefcase (BiB) is a portable sensor network platform that is trivially...