Tohme

2013–2014 7

Project Description

Tohme combines machine learning, computer vision (CV), and custom crowd interfaces to find curb ramps remotely in GSV scenes. Tohme consists of two workflows, a human labeling pipeline and a CV pipeline with human verification, which are scheduled dynamically based on predicted performance. Using 1,086 GSV scenes (street intersections) from four North American cities and data from 403 crowd workers, we show that Tohme performs similarly in detecting curb ramps compared to a manual labeling approach alone (F-measure: 84% vs. 86% baseline) but at a 13% reduction in time cost. Our work contributes the first CV-based curb ramp detection system, a custom machine-learning based workflow controller, a validation of GSV as a viable curb ramp data source, and a detailed examination of why curb ramp detection is a hard problem along with steps forward.

This project is part of our larger research agenda in combining crowdsourcing, computer vision, and online map imagery to transform how we collect data about street-level accessibility.

Publications

Tohme: Detecting Curb Ramps in Google Street View Using Crowdsourcing, Computer Vision, and Machine Learning

Kotaro Hara, Jin Sun, Robert Moore, David Jacobs, Jon E. Froehlich

Proceedings of UIST 2014

keywords: crowdsourcing accessibility, computer vision, google street view, amazon mechanical turk

PDF | doi | Citation | Project Sidewalk • Tohme

An Initial Study of Automatic Curb Ramp Detection with Crowdsourced Verification using Google Street View Images

Kotaro Hara, Jin Sun, Noa Chazan, David Jacobs, Jon E. Froehlich

Poster Proceedings of HCOMP 2013

keywords: computer vision, accessible urban navigation, mechanical turk, data collection, accessible cities, physical accessibility, crowdsourcing, google streetview, crowd verification

PDF | doi | Citation | Project Sidewalk • Tohme