RASSAR

2022–Present Explore the Code 9

ARTennis

2023–Present
CookAR

2023–Present
HoloSound

2020
AR Captioning

2016–2022
AR Magnification

2016–2018

Project Description

To help improve the safety and accessibility of indoor spaces, researchers and health professionals have created assessment instruments that enable homeowners and trained experts to audit and improve homes. With advances in computer vision, augmented reality (AR), and mobile sensors, new approaches are now possible. We introduce RASSAR (Room Accessibility and Safety Scanning in Augmented Reality), a new proof-of-concept prototype for semi-automatically identifying, categorizing, and localizing indoor accessibility and safety issues using LiDAR + camera data, machine learning, and AR. We present an overview of the current RASSAR prototype and a preliminary evaluation in a single home.

Publications

RASSAR: Room Accessibility and Safety Scanning in Augmented Reality

Xia Su, Han Zhang, Kaiming Cheng, Jaewook Lee, Qiaochu (Steve) Liu, Wyatt Olson, Jon E. Froehlich

Proceedings of CHI 2024 | Acceptance Rate: 26.3% (1060 / 4028)

keywords: computer vision, augmented reality, 3D reconstruction, indoor accessibility auditing, lidar

PDF | doi | Citation | Code | RASSAR

A Demonstration of RASSAR: Room Accessibility and Safety Scanning in Augmented Reality

Xia Su, Kaiming Cheng, Han Zhang, Jaewook Lee, Wyatt Olson, Jon E. Froehlich

Extended Abstract Proceedings of ASSETS 2023 | Acceptance Rate: 51.9% (40 / 77)

keywords: accessibility, computer vision, indoor accessibility auditing, lidar, room reconstruction

PDF | doi | Citation | RASSAR

Towards Semi-automatic Detection and Localization of Indoor Accessibility Issues using Mobile Depth Scanning and Computer Vision

Xia Su, Kaiming Cheng, Han Zhang, Jaewook Lee, Yueqian Zhang, Jon E. Froehlich

UrbanAccess 2022

keywords: accessibility, computer vision, augmented reality, 3D reconstruction, indoor accessibility auditing

PDF | doi | Citation | RASSAR

Talks

AI-Powered AR Accessibility

Aug. 8, 2024 | Apple's HCML Workshop

Cupertino, CA

Jon E. Froehlich

PDF | PPTX | CookAR • DIAM: Drone-based Indoor Accessibility Mapping • RASSAR • ARTennis • GazePointAR

RASSAR: Room Accessibility and Safety Scanning in Augmented Reality

May 15, 2024 | CHI 2024

Honolulu, Hawaiʻi

PDF | PPTX | RASSAR

We design, build, and evaluate new interactive tools and techniques to address pressing societal challenges. Makeability refers both to how our technological innovations make new abilities possible for humans as well as our educational mission to help students gain new abilities as they learn and grow through research, invention, and human-centered design.