StreetReaderAI

2024–Present 5

RampNet

2025–Present
Accessibility Scout

2024–Present
AltGeoViz

2024–Present
Accessibility for Whom?

2024–Present
AccessParkCV

2023–Present

Project Description

Interactive streetscape mapping tools such as Google Street View (GSV), Meta Mapillary, and Apple Look Around enable users to virtually navigate and experience real-world environments via immersive 360° imagery but remain fundamentally inaccessible to blind users. We introduce StreetReaderAI, the first-ever accessible street view tool, which combines context-aware, multimodal AI, accessible navigation controls, and conversational speech. With StreetReaderAI, blind users can virtually examine destinations, engage in open-world exploration, or virtually tour any of the over 220 billion images and 100+ countries where GSV is deployed. We iteratively designed StreetReaderAI with a mixed-visual ability team and performed an evaluation with eleven blind users. Our findings demonstrate the value of an accessible street view in supporting POI investigations and remote route planning.

Publications

Making Street View Accessible Using Context-Aware Multimodal AI: A Demo of StreetReaderAI

Jon E. Froehlich, Alexander J. Fiannaca, Nimer Jaber, Victor Tsaran, Shaun K. Kane

Proceedings of ASSETS 2025 | Acceptance Rate: 66.1% (82 / 124)

keywords: google street view, multimodal llm (mllm), accessible street view

PDF | doi | Citation | StreetReaderAI

StreetReaderAI: Making Street View Accessible Using Context-Aware Multimodal AI

Jon E. Froehlich, Alexander J. Fiannaca, Nimer Jaber, Victor Tsaran, Shaun K. Kane

Proceedings of UIST 2025 | Acceptance Rate: 22.2% (210 / 947)

keywords: blind users, accessible maps, multimodal llm (mllm), accessible street view

PDF | doi | Citation | StreetReaderAI

Videos

StreetReaderAI: Making Street View Accessible Using Context-Aware Multimodal AI

YouTube | StreetReaderAI

StreetViewAI: Making Street View Accessible Using Context-Aware Multimodal AI (30-Sec Preview)

YouTube | StreetReaderAI

Talks

StreetReaderAI: Making Street View Accessible Using Context-Aware Multimodal AI

Sept. 28, 2025 | UIST

Busan, Korea

Jon E. Froehlich, Alexander J. Fiannaca, Nimer Jaber, Victor Tsaran, Shaun K. Kane

PDF | PPTX | StreetReaderAI

We design, build, and evaluate new interactive tools and techniques to address pressing societal challenges. Makeability refers both to how our technological innovations make new abilities possible for humans as well as our educational mission to help students gain new abilities as they learn and grow through research, invention, and human-centered design.