Tohme

Quick Info

Project Date Jan. 1, 2013 - Oct. 8, 2014
Sponsors: NSF
Keywords: crowdsourcing accessibility, computer vision, google streetview, urban accessibility

About

A workflow diagram depicting Tohme’s four main sub-systems.

Tohme uses machine learning to predict the quality of computer vision output and routes work accordingly.

Tohme combines machine learning, computer vision (CV), and custom crowd interfaces to find curb ramps remotely in GSV scenes. Tohme consists of two workflows, a human labeling pipeline and a CV pipeline with human verification, which are scheduled dynamically based on predicted performance. Using 1,086 GSV scenes (street intersections) from four North American cities and data from 403 crowd workers, we show that Tohme performs similarly in detecting curb ramps compared to a manual labeling approach alone (F-measure: 84% vs. 86% baseline) but at a 13% reduction in time cost. Our work contributes the first CV-based curb ramp detection system, a custom machine-learning based workflow controller, a validation of GSV as a viable curb ramp data source, and a detailed examination of why curb ramp detection is a hard problem along with steps forward.

This project is part of our larger research agenda in combining crowdsourcing, computer vision, and online map imagery to transform how we collect data about street-level accessibility.

Publications

Tohme: Detecting Curb Ramps in Google Street View Using Crowdsourcing, Computer Vision, and Machine Learning

Kotaro Hara, Jin Sun, Robert Moore, David Jacobs, Jon E. Froehlich

Proceedings of UIST 2014

keywords: crowdsourcing accessibility, computer vision, google street view, amazon mechanical turk

PDF | doi | Citation | Project Sidewalk • Tohme

An Initial Study of Automatic Curb Ramp Detection with Crowdsourced Verification using Google Street View Images

Kotaro Hara, Jin Sun, Noa Chazan, David Jacobs, Jon E. Froehlich

Poster Proceedings of HCOMP 2013

keywords: computer vision, accessible urban navigation, mechanical turk, data collection, accessible cities, physical accessibility, crowdsourcing, google streetview, crowd verification

PDF | doi | Citation | Project Sidewalk • Tohme

Exploring Early Solutions for Automatically Identifying Inaccessible Sidewalks in the Physical World using Google Street View

Kotaro Hara, Victoria Le, Jin Sun, David Jacobs, Jon E. Froehlich

HCIC2013 Workshop

keywords: computer vision, google street view, machine learning, crowdsourcing, physical world accessibility, street-level accessibility

PDF | Citation | Project Sidewalk • Tohme

Quick Info

About

Publications

Recent News

Links