Flow: Traffic simulation joins forces with deep reinforcement learning and cloud computing

January 11, 2019
  • Aimsun and UC Berkeley’s Institute of Transportation Studies have formed a new partnership
  • The teams have worked together to produce Flow, a tool for managing large-scale traffic systems with a mix of human-driven and autonomous vehicles
  • Flow is the first open source architecture to integrate microsimulation tools with state-of-the-art deep reinforcement learning libraries in the cloud
  • Flow provides benchmarks in the use of deep reinforcement learning to create controllers for mixed-autonomy traffic, where connected and autonomous vehicles interact with human drivers and infrastructure

UC Berkeley’s Institute of Transportation Studies and Aimsun recently created a new partnership to release Flow, a tool for managing large-scale traffic systems that feature a mix of human-driven and autonomous vehicles.

Originally launched in September 2018, Flow is now integrated with Aimsun Next to become the first open source architecture to integrate microsimulation tools with state-of-the-art deep reinforcement learning libraries in the cloud.

“Working harmoniously with others, be those professional relationships or software interfaces, has always been a core value at Aimsun,” says Alex Gerodimos, President, Aimsun Inc. “Our ongoing collaboration with UC Berkeley exemplifies this philosophy perfectly at both levels: our decision to lend our direct and complete support to this project was very deliberate and we are more than happy that Next users will be able to benefit immediately from these new machine learning libraries as a result.”

Recent developments in multi-agent deep reinforcement learning algorithms, advancements in Aimsun Next microscopic traffic simulation libraries, and improved ease of cloud computing like Amazon Web Services (AWS), together provide a powerful ecosystem for finding near-optimal decision-making rules such as AV acceleration, AV lane changing, or traffic signal splits in mixed autonomy systems.

“In mixed-autonomy traffic control, evaluating machine learning methods is challenging due to the lack of standardized benchmarks,” says Alexandre Bayen, Director, ITS Berkeley. “Systematic evaluation and comparison will not only further our understanding of the strengths of existing algorithms but also reveal their limitations and suggest directions for future research.”

Flow is a traffic control framework that provides a suite of pre-built traffic control scenarios, tools for designing custom traffic scenarios, and integration with deep reinforcement learning libraries such as RLlib and traffic microsimulation libraries, which can be used to apply deep reinforcement learning breakthroughs to various cases in traffic management, which involve classical traffic infrastructure (traffic lights, metering, etc.) and mobile infrastructure (mixed autonomy traffic, in particular using connected and automated vehicles to regulate traffic).

Flow enables users to build modular traffic-scenarios which can be combined to tackle complex situations. These building-blocks help break a problem down into smaller, tractable pieces that can be composed as controllers for new scenarios. For instance, single-lane/multi-lane and merge building-blocks can be used to study stop-and-go and merging traffic behaviors along a highway. https://flow-project.github.io/

The team will be presenting Flow at the Aimsun stand at the Transportation Research Board Annual Meeting 2019, Washington DC.