The rapid development of the internet of things (IoT), sensing technologies, Artificial Intelligence (AI), machine learning and deep learning techniques, have yielded new perspectives on how novel technologies can be applied to smart cities. The New York City IoT Strategy report 1 highlighted that the city has been home to a major expansion in IoT and AI use in the last decade, with impacts on many areas, including its transportation system.
As a subfield of AI, Computer Vision is showing promising potential in understanding the realistic dynamics of cities. While cities are complex by nature, especially cities like NYC, the applications of computer vision show progress in tackling a variety of complex physical and non-physical visual tasks. In addition, computer vision can turn existing infrastructure into smart sensors in a myriad of ways, and new applications are being continuously developed. Agencies including NYC Department of Transportation (NYC DOT), NYC Department of Design and Construction (NYC DDC), and NYC Mayor’s Office of the Chief Technology Officer (CTO) have identified a “Wishlist” of area of interests in using computer vision technologies for the City.
The Wishlist include using computer vision tools to validate the accuracy of collected mobility data (e.g., turning movement counts, vehicle classifications, vehicle and pedestrian speed, etc.), parking utilization, work zone assessment, incident detection, mobility aids detection, vehicle- pedestrian conflicts as well as assessing how the computer vision algorithm can be trained for NYC conditions where there may be difficult sightlines and blockages and restrictions on camera placement. This “Wishlist” provides the foundation of understanding what can the technology offer and what are the needs from the agencies.
This project is focused on developing a deep learning based data acquisition and analytics tool using vision-based sensors (i.e., cameras) to understand cities with machine eyes. The team will assess the maturity of various smart city applications using computer vision and object detection (e.g., pedestrian detection, work zone identification, curb lane usage, connected and automated vehicles [CAVs]) as well as the needs of the local agencies. The goal is to demonstrate the cost-effectiveness of the computer vision technology to generate new stream of mobility data and provide support for planning and operational strategies, utilizing both existing transportation infrastructure and emerging probe and CAVs.
More specifically, this project aims to establish an inventory of available traffic camera systems in the U.S. and deploy two computer vision smart city applications based on stakeholder feedback that are customized for New York City (NYC). The team will also establish a formalized pipeline for running the computer vision algorithms enhanced for NYC conditions and prototype the applications for real-world implementation.
Senior Research Associate, NYU
Jingqin Gao is the Principal Investigator on this project.
Kaan Ozbay is a Co-Principal Investigator on this project.