Lane Changing of Autonomous Vehicles in Mixed Traffic Environments: A Reinforcement Learning Approach


According to an accident report from Volvo, lane changing maneuver is the main cause of various severe highway accidents, due to human drivers’ inaccurate estimation and prediction of the surrounding traffic, illegal maneuver, and inefficient driving skill. Autonomous driving is regarded as a solution to reduce these human errors. At present, there are many obstacles to developing automated lane-changing technology, including dynamically changing interactions between vehicles, complex routing choices, and the interactions between vehicles and the environment.

In this proposal, in order to guarantee the safety of autonomous vehicles (AV), improve passenger comfort, and increase traffic efficiency, we aim to develop innovative learning-based control methods for lane changing of connected and autonomous vehicles (CAVs) in mixed traffic by a combined use of reinforcement learning and optimal control techniques.

Conventionally, automated lane changing consists of three phases: surrounding environment detection and prediction, trajectory planning, and vehicle control. With the predicted states of the surrounding vehicles and the environment, the aim of the trajectory planning module is to plan a safe trajectory from the CAV’s original position in the current lane to the desired position in the target lane.

For the low-level vehicle control, the aim is to design a trajectory tracking controller to track the aforementioned trajectory. Most of the existing controllers are developed by model-based approaches, such as sliding mode control and model predictive control, where the accurate dynamic model of the CAV is indispensable. However, system modeling is time-consuming, costly, and often involves modeling errors. A controller designed based on an inaccurate model may destabilize the vehicle system, which threatens the safety of the CAV and its surrounding traffic. Robust control can stabilize the vehicle system if there are errors or uncertainties in the model, but this compromises the system’s
transient response.

Research Objectives

In this proposal, we will extend our recent work from lane-keeping to lane changing, by developing data-driven adaptive learning algorithms to achieve the optimal lane-changing maneuver in the mixed traffic. Novel learning-based vehicle lane changing techniques will be proposed based on reinforcement learning and optimal control techniques. Once the lane changing task is completed, the CAV automatically switches to our previously developed learning-based controls for lane keeping.

Simulation of Urban MObility (SUMO) simulation platform will be used while the trajectories of HDVs in the mixed traffic will be obtained from the next generation simulation (NGSIM) or similar realistic datasets with actual HDV trajectories to demonstrate that our results are not a simple artifact of the simulation models that are being used.


Kaan Ozbay

Director, C2SMART
Professor, NYU

Kaan Ozbay is the Principal Investigator on this project.

Zhong-Ping Jiang

Professor, NYU

Zhong-Ping Jiang is the Principal Investigator on this project.

Tong Liu

Researcher, NYU

Tong Liu is a Researcher on this project.

Leilei Cui

Researcher, NYU

Leilei Cui is a Researcher on this project.

Sayan Chakraborty

Researcher, NYU

Sayan Chakraborty is a Researcher on this project.