Project Description

Date

2016–Present
Recent work has applied machine learning methods to automatically find and/or assess pedestrian infrastructure in online map imagery (e.g., satellite photos, streetscape panoramas). While promising, these methods have been limited by two interrelated issues: small training sets and the choice of machine learning model. In this paper, aided by the recently released Project Sidewalk dataset of 300,000+ image-based sidewalk accessibility labels, we present the frst examination of deep learning to automatically assess sidewalks in Google Street View (GSV) panoramas. Specifcally, we investigate two application areas: automatically validating crowdsourced labels and automatically labeling sidewalk accessibility issues. For both tasks, we introduce and use a residual neural network (ResNet) modifed to support both image and non-image (contextual) features (e.g., geography). We present an analysis of performance, the effect of our non-image features and training set size, and cross-city generalizability. Our results significantly improve on prior automated methods and, in some cases, meet or exceed human labeling performance.

Funding

Publications

Scaling Crowd+AI Sidewalk Accessibility Assessments: Initial Experiments Examining Label Quality and Cross-city Training on Performance

Michael Duan, Sho Kiami, Logan Milandin, Johnson Kuang, Mikey Saugstad, Maryam Hosseini, Jon E. Froehlich

Poster Proceedings of ASSETS'22 | Acceptance Rate: 58.9% (43 / 73)

PDF | doi | Citation | Project Sidewalk • Deep Learning for Sidewalk Assessment

Towards Global-Scale Crowd+AI Techniques to Map and Assess Sidewalks for People with Disabilities

Maryam Hosseini, Mikey Saugstad, Fabio Miranda, Andres Sevtsuk, Cláudio T. Silva, Jon E. Froehlich

CVPR2022 Workshop: Accessibility, Vision, and Autonomy (AVA)

PDF | doi | Citation | Project Sidewalk • Deep Learning for Sidewalk Assessment

Deep Learning for Automatically Detecting Sidewalk Accessibility Problems Using Streetscape Imagery

Galen Weld, Esther Jang, Anthony Li, Aileen Zeng, Kurtis Heimerl, Jon E. Froehlich

Proceedings of ASSETS 2019 | Acceptance Rate: 25.9% (41 / 158) | Best Paper Award

PDF | doi | Citation | Code | Project Sidewalk • Deep Learning for Sidewalk Assessment

Talks
Project Sidewalk: Crowd+AI Techniques to Map & Assess the World's Sidewalks

Feb. 29, 2024 | NSF Smart and Connected Communities Panel on “Pathways to Transitioning Project Outcomes"

Nashville, Tennessee

PDF | PPTX | Project Sidewalk + Community Science | Project Sidewalk | Deep Learning for Sidewalk Assessment

Project Sidewalk: Mapping the Accessibility of the Physical World at Scale using Interactive Computational Tools

Jan. 22, 2020 | Google Tech Talk

Seattle, WA

PDF | PPTX | Project Sidewalk | Urban Accessibility Evolution | Transportation Analytics | Deep Learning for Sidewalk Assessment | AccessVis | Accessibility-Infused Maps

Deep Learning for Automatically Detecting Sidewalk Accessibility Problems Using Streetscape Imagery

Oct. 29, 2019 | ASSETS'19

Pittsburgh, PA

PDF | PPTX | Deep Learning for Sidewalk Assessment

Related Projects

This is an icon indicating the system diagram for LabelAid
LabelAId
2023–Present
A screenshot of BusStop CV showing automatic detections
BusStopCV
2023–Present

We design, build, and evaluate new interactive tools and techniques to address pressing societal challenges. Makeability refers both to how our technological innovations make new abilities possible for humans as well as our educational mission to help students gain new abilities as they learn and grow through research, invention, and human-centered design.

Recent News

April 18, 2024

Jon Gives Invited Talk at AI2

March 19, 2024

UW News: Improving Mobility Technologies for Children

Feb. 29, 2024

Froehlich Gives Invited Talk on Project Sidewalk at NSF Smart and Connected Communities PI Meeting

Links