Learning Optimal Traffic Routing Behaviors Using Markovian Framework in Microscopic Simulation

This article applies the existing Markovian traffic assignment framework to novel traffic control strategies. In the Markovian traffic assignment framework, transition matrices are used to derive the traffic flow allocation. In contrast to the static traffic assignment, the framework only requires flow split ratio at every intersection, bypassing the need of computing path flow allocation. Consequently, compared to static traffic assignment, drivers’ routing behaviors can be modeled with fewer variables. As a result, it could be used to improve the efficiency of traffic management, especially in large scale applications. To begin with, the article introduces Markovian traffic assignment and connects it to the classic static traffic assignment. Then, the framework is extended to dynamic traffic assignment using microscopic traffic simulator Simulation of Urban Mobility (SUMO). In a case study, the framework is applied to a standard benchmark network, where optimal routing behaviors are independently learned through grid search, random search, and evolution strategies, under three different reward functions (network outflow, total vehicle hours of travel, and average marginal regret). The case study shows that the this novel traffic control strategy is promising, as Markov chain theory supports the ability to scale up to larger networks.