Deep Learning for Sidewalk Assessment

Quick Info

Project Date Jan. 1, 2016 - Present
Sponsors: NSF
Keywords: accessible cities, machine learning, street-level accessibility, urban accessibility, deep learning, sidewalks

News

Makeability Lab Collaborator Maryam Hosseini Defends Her PhD Dissertation

Aug 17, 2022 | Jon

Congratulations to Maryam Hosseini on their successful PhD defense on semi-automatic sidewalk assessments. Wonderful work. Terrific presentation. Beautiful slides. Our work together played but a small role in this incredibly impactful research: Hosseini, M., Saugstad, M., Miranda, F., Sevtsuk, A., Silva, C. T., Froehlich, J. E. (2022). Towards Global-Scale Crowd+AI Techniques to Map and Assess Sidewalks for People with Disabilities. CVPR2020 Workshop: Accessibility, Vision, and Autonomy (AVA). DOI: https://doi.org/10.48550/arXiv.2206.13677 Congratulations Maryam!

Spatial Data Science Symposium: Future of Global-Scale Data Collection and Analysis on Urban (in)Accessibility for People with Disabilities

Dec 13, 2021 | Jon

Together with Professor Fabior Miranda from the University of Illinois, Chicago and Maryam Hosseini from Rutgers/NYU, we ran a mini-symposium session on the The Future of Global-Scale Spatial Data Collection and Analyses on Urban (in)Accessibility for People with Disabilities at the 2nd Spatial Data Science Symposium 2021. In our session, we brought together experts in disability, human mobility, urban planning, and computer science to discuss state-of-the-art methods for measuring the quality, condition, and accessibility of urban infrastructure, how these methods may enable new types of geospatial analysis and visualization, and the possibilities for data-driven policy change and accessible urban development. Our overarching goal was to identify open challenges, share current work across disciplines, and spur new collaborations. We had over 50 participants join and a set of lightning talks from Anat Caspi and Nick Bolten from UW, Roberto M. Cesar Jr. and Eric K. Tokuda, from the University of São Paulo, Holger Dieterich and Sebastian Felix Zappe from Sozialhelden, Victor Pineda from the Inclusive Cities Lab and Worldenabled.org, Yochai Eisenberg from the University of Illinois, Chicago, and Andres Sevtsuk from UW. Froehlich, J. E., Miranda, F., Hosseini, M., Bolten, N., Caspi, A., Cesar Jr., R. M., Dieterich, H., Eisenberg, Y., Pineda, V., Saha, M., Saugstad, M., Sevtsuk, A., Silva, C. T., Tokuda, E. K., Zappe, S. F. (2021). The Future of Global-Scale Spatial Data Collection and Analyses on Urban (in)Accessibility for People with Disabilities. Spatial Data Science Symposium 2021.

Manaswi Gives Google Tech Talk on Project Sidewalk

Jan 22, 2020 | Jon

Manaswi Saha gave an invited Google Tech Talk on Project Sidewalk and interactive accessibility geo-visualizations. The talk slides are here.

Deep Learning for Sidewalk Accessibility Awarded 'Best Student Paper' at ASSETS'19

Oct 29, 2019 | Jon

Our ASSETS'19 paper "Deep Learning for Automatically Detecting Sidewalk Accessibility Problems Using Streetscape Imagery" was just recognized with the 'Best Student Paper Award'--given to only one of the 158 submissions Congrats team!

Two ASSETS'19 Papers Nominated for Best Paper

Oct 28, 2019 | Jon

Both of our ASSETS'19 papers were nominated for 'Best Paper' at ASSETS'19: Weld, G., Jang, E., Li, A., Zeng, A., Heimerl, K., Froehlich, J. E. (2019). Deep Learning for Automatically Detecting Sidewalk Accessibility Problems Using Streetscape Imagery. Proceedings of ASSETS 2019. Jain, D., Desjardins, A., Findlater, L., Froehlich, J. E. (2019). Autoethnography of a Hard of Hearing Traveler. Proceedings of ASSETS 2019. Congrats to all authors!

About

Our Deep Learning Techniques Classify Four Label Types

Examples of the four label types used to train and test our deep learning models for semi-automatic sidewalk assessment: curb ramps, missing curb ramps, obstructions, and surface problems.

Recent work has applied machine learning methods to automatically find and/or assess pedestrian infrastructure in online map imagery (e.g., satellite photos, streetscape panoramas). While promising, these methods have been limited by two interrelated issues: small training sets and the choice of machine learning model. In this paper, aided by the recently released Project Sidewalk dataset of 300,000+ image-based sidewalk accessibility labels, we present the frst examination of deep learning to automatically assess sidewalks in Google Street View (GSV) panoramas. Specifcally, we investigate two application areas: automatically validating crowdsourced labels and automatically labeling sidewalk accessibility issues. For both tasks, we introduce and use a residual neural network (ResNet) modifed to support both image and non-image (contextual) features (e.g., geography). We present an analysis of performance, the effect of our non-image features and training set size, and cross-city generalizability. Our results significantly improve on prior automated methods and, in some cases, meet or exceed human labeling performance.