Project Description

Tohme combines machine learning, computer vision (CV), and custom crowd interfaces to find curb ramps remotely in GSV scenes. Tohme consists of two workflows, a human labeling pipeline and a CV pipeline with human verification, which are scheduled dynamically based on predicted performance. Using 1,086 GSV scenes (street intersections) from four North American cities and data from 403 crowd workers, we show that Tohme performs similarly in detecting curb ramps compared to a manual labeling approach alone (F-measure: 84% vs. 86% baseline) but at a 13% reduction in time cost. Our work contributes the first CV-based curb ramp detection system, a custom machine-learning based workflow controller, a validation of GSV as a viable curb ramp data source, and a detailed examination of why curb ramp detection is a hard problem along with steps forward.

This project is part of our larger research agenda in combining crowdsourcing, computer vision, and online map imagery to transform how we collect data about street-level accessibility.


Tohme: Detecting Curb Ramps in Google Street View Using Crowdsourcing, Computer Vision, and Machine Learning

Kotaro Hara, Jin Sun, Robert Moore, David Jacobs, Jon E. Froehlich

Proceedings of UIST 2014

An Initial Study of Automatic Curb Ramp Detection with Crowdsourced Verification using Google Street View Images

Kotaro Hara, Jin Sun, Noa Chazan, David Jacobs, Jon E. Froehlich

Poster Proceedings of HCOMP 2013

Exploring Early Solutions for Automatically Identifying Inaccessible Sidewalks in the Physical World using Google Street View

Kotaro Hara, Victoria Le, Jin Sun, David Jacobs, Jon E. Froehlich

HCIC2013 Workshop