As car companies increasingly tout semi- and fully-autonomous features — including lane control and “autopilot” — and 29 states have enacted legislation related to self-driving vehicles, UC Berkeley transportation researchers are addressing this emerging era of smart vehicles with a tool that uses machine learning to manage traffic where autonomous, semi-autonomous and manned vehicles share the road. The project, called Flow, rolled out its first proposed standards for solving real-world traffic problems, including easing bottlenecking on the San Francisco-Oakland Bay Bridge, today (Monday, Oct. 29) at the Conference on Robotic Learning in Zurich, Switzerland.
Many traffic researchers are addressing smart-vehicle integration, but compared to models that use manually derived algorithms to design controls like metering-light timing, machine-learning-based controls can provide benefits like lower energy consumption and novel traffic-management solutions that are out of reach of human calculations.
“Flow solves large-scale, multi-vehicle problems by using simulations that are much more efficient than what can be produced without the aid of artificial intelligence,” said electrical engineering and computer sciences professor Alexandre Bayen, director of the UC Berkeley Institute of Transportation Studies and the study’s principal investigator. “And we’ve made it a cloud-based, open-source system so the development community can continue to build on it.”
A novel feature of the system is automated cars using data from nearby smart vehicles or infrastructure to manage traffic, effectively becoming mobile traffic-managing robots. For example, to prevent bottlenecking, an automated car could use its speed and position to control nearby vehicles as they merge. Or it could pace its speed to help prevent the random, human-caused slowdowns that increase travel time and frustrate drivers.
“We need a more sophisticated traffic-management system, and we also need to think about how we can use these autonomous vehicles as part of traffic control,” said Eugene Vinitsky, co-lead author on the study with Aboudy Kreidieh, both Ph.D. students in Bayen’s lab. “There are huge benefits that even 4 or 5 percent of vehicles can provide to the rest of the traffic on the road.”
Flow uses deep reinforcement learning, a facet of machine learning that continuously improves a system’s decision-making by learning from each problem it solves, then advancing those solutions through numerous iterations. Many applications, including robotics and game theory, use deep reinforcement learning, but this is the first time it has been integrated with traffic-simulation tools.
An apples-to-apples framework
The study specifies highly detailed scenarios — standard “tasks” that engineers can use to solve common types of traffic challenges like bottlenecks and intersection control. The solutions become shared baselines, called benchmarks, that are critical to making progress, researchers say.
“Unless we’re working on the same problem, it’s hard to compare results. Are you looking at a New York highway or a California freeway? A group of 20 cars or 50? You need an apples-to-apples comparison to understand which solution works better,” Vinitsky said.
Berkeley engineers have devloped a system that brings machine learning to traffic management, particularly where autonomous, semi-autonomous and manned vehicles share the road. (iStock)Daniel Work, associate professor of civil and environmental engineering at Vanderbilt University, who studies these issues but is not associated with the Flow project, said having the code available is critical. “In the transportation community we can use these benchmarks to compare and compete, and most importantly, to reproduce results,” Work said. The scenarios also accelerate progress by removing the need for developers to create their own elaborate models, he added. “Engineers can stop recreating the same task — like an on-ramp merge — over and over. Instead, we can focus our efforts just on finding solutions to that task.”
Standardized benchmarks already exist in other deep reinforcement learning applications, including natural-language processing and robotics. But for traffic management, researchers were unable to find such standards, so they set out to create them. Earlier papers from Bayen’s lab presented Flow as a traffic-simulation platform, then established initial benchmarks using simple scenarios such as cars driving on a ring or figure eight.
“The benchmarks we presented today are the first time we’ve applied this work to more complex, real-world scenarios,” Vinitsky said.
In one task — featuring a computer model of the San Francisco-Oakland Bay Bridge, a classic bottleneck where 15 toll lanes merge into five traffic lanes — the goal is to maximize how many cars flow off the bridge.
“Now that we’ve created a set of details for that task — the traffic volume and vehicle position and speed in each lane — anyone can come up with a solution, and there’s a standard for comparing those solutions,” Vinitsky said. “And, obviously, Bay Bridge traffic is a pertinent issue.”
The study’s other benchmarks include an on-ramp merge and a model of a Manhattan grid of traffic lights.
Researchers plan to tackle increasingly complex scenarios, and they hope outside collaborators will do the same. The end goal, Vinitsky said, is for the system to manage traffic at the citywide scale, incorporating the richness of human driving while capturing the benefits of the small but growing percentage of smart vehicles on the road.
Researchers also plan to study potential downsides and unintended consequences of this technological approach. For example, if traffic smoothing works too well, improved travel times could lure more drivers onto the road, undermining short-term gains as well the larger goals of reducing greenhouse gas emissions and energy consumption.
Understanding both the opportunities and the pitfalls of mixed autonomy traffic is an incremental step to the gradual — and inevitable — integration of autonomous vehicles in not-too-distant future, Bayen said. “We can simulate what this future will look like, and we can start working on strategies to improve it.”