Work Zone Safety III: Calibration of Safety Notifications through Reinforcement Learning and Eye Tracking

Overview

According to the Federal Highway Administration (FHWA), work zone fatalities at road construction projects account for up to 3% of all workplace fatalities in a given year [1], and the primary causes are runovers/backovers, collisions, and caught in-between mobile equipment. Hence, drivers and the way they perceive the work zone and related notifications are primary factors required to reduce fatalities. A study of work zone crash data in five states showed that around half of the crashes occur within or adjacent to work activities, putting workers in danger together with drivers [2]. To reduce work zone injuries and fatalities, regulations such as mandated Personal Protective Equipment (PPE), traffic control plans, advance warning signs, the share of traveler information, and signal timing adjustments (ANSI, OSHA) were introduced by the regulatory bodies. However, these mainly aim for changing the behavior of drivers instead of workers. Although there is a large body of analysis and modeling literature related to work zone accidents as documented in [3], the actual safety treatments applicable to real-world work zones are limited at best and there is still a need for proactive approaches to be deployed at highway work zones, capable of warning construction workers of approaching hazards in advance. 

To improve work zone safety, in the previous two phases of this project, we proposed a virtual reality (VR)-based platform that integrates with SUMO and hardware in the loop sensors to realistically simulate dangerous situations in work zones (i.e., enabling worker-initiated changes in the work zone to be accounted in SUMO and updated simulation to be displayed real-time in VR). In this phase, we propose to add two main components to the existing VR work zone safety testing platform. The first component focuses on monitoring construction workers’ attention. To that end, we propose adding new functionality to the current VR platform to track the subjects’ attention through his/her head-movement and eye-movement to infer his/her gaze pattern. With the introduction of this method to measure the subject’s attention, we plan to capture additional critical information about the decision a worker makes.

Research Objectives

In the third phase of this work zone safety project, we aim to calibrate the frequency, timing, and modality of alarms through (1) the introduction of an eye-tracking component to the existing VR platform, and (2) the introduction of a reinforcement learning-based model to optimize the calibration of alarms. 

Specifically, we have the following research objectives:

  • Understand the construction workers’ attention to work zone conditions (e.g., vehicle head direction/speed toward work zone, worker vicinity to work zone perimeters) when an alarm is delivered by tracking the head-movement and eye-movement of subjects in VR.
  • Evaluate the effectiveness of alarms sent in different modalities, frequencies and timings to the workers and determine the key factors that influence the reactions of workers to alarms.
  • Develop an alarm delivery system that is tailored to maximize workers’ attention to the alarms sent regarding dangerous situations using reinforcement learning.

Deliverables for this phase of the project will include:

  • An implementation of an attendance monitoring system embedded in the current VR platform to measure subjects’ gaze patterns when a dangerous situation is simulated in the VR.
  • The evaluation of whether data arriving with different delays/frequency from the sensors and wearables are meant to correlate with human responses to notifications.
  • The development of an alarm delivery system that is trained to maximize the attention of the workers towards alarms sent regarding potentially dangerous situations in the work zone. 
  • A report describing the RL-based model for calibration of notifications, and worker attention level to dangerous situations in highway work zones. 

Personnel

Dan Lu

PhD candidate at New York University Tandon School of Engineering

Dan Lu is a Researcher on this project.

Semiha Ergan

Associate Professor at New York University

Semiha Ergan is a Principal Investigator on this project.

Suzana Duran Bernardes

Ph.D. student at New York University Tandon School of Engineering

Suzana Duran Bernardes is a Researcher on this project.

Keundeok Park

PhD candidate at New York University Tandon School of Engineering

Keundeok Park is a Researcher on this project.

Kaan Ozbay

Director of the C2SMART Center

Kaan Ozbay is a Principal Investigator on this project.

Details