ITS Berkeley

Hierarchies of temporally decoupled policies present a promising approach for enabling structured exploration in complex long-term planning problems. To fully achieve this approach an end-to-end training paradigm is needed. However, training these multi-level policies has had limited success due to challenges arising from interactions between the goal-assigning and goal-achieving levels within a hierarchy. In this article, we consider the policy optimization process as a multi-agent process. This allows us to draw on connections between communication and cooperation in multi-agent RL, and...

A wide range of reinforcement learning (RL) problems --- including robustness, transfer learning, unsupervised RL, and emergent complexity --- require specifying a distribution of tasks or environments in which a policy will be trained. However, creating a useful distribution of environments is error prone, and takes a significant amount of developer time and effort. We propose Unsupervised Environment Design (UED) as an alternative paradigm, where developers provide environments with unknown parameters, and these parameters are used to automatically produce a distribution over valid,...

Introduction Informal caregiving is an essential element of health-care delivery. Little data describes how caregivers structure care recipients’ lives and impact their functional status. Methods We performed observational studies of community dwelling persons with dementia (PWD) to measure functional status by simultaneous assessment of physical activity (PA) and lifespace (LS). We present data from two caregiver/care-recipient dyads representing higher and average degrees of caregiver involvement. Results We acquired >42,800 (subject 1); >41,300 (subject 2) PA data points and >...

This paper presents the problem of block simplex constrained signal recovery, which has been demonstrated to be a suitable formulation for estimation problems in networks such as route flow estimation in traffic. There are several natural approaches to this problem: compressed sensing, Bayesian inference, and convex optimization. This paper presents new methods within each framework and assesses their respective abilities to reconstruct signals, with the particular emphasis on sparse recovery, ability to incorporate prior information, and scalability. We then apply these methods to route...