Project Description

2018–2022
We introduce SoundWatch, an Android app that provides glanceable, always-available sound feedback on smartwatches for d/Deaf and hard-of-hearing users. Created by a diverse team at University of Washington and led by a hard-of-hearing innovator who wanted to build for his community, SoundWatch uses machine learning to alert its users to important sounds occurring in their environment. With everything from critical fire alarms to speech alerts, users know they can rely on SoundWatch to make their environment more accessible. In the spirit of innovation and the spread of knowledge, SmartWatch is freely available and open-sourced for anyone wanting to learn more and contribute to its development. Since Dhruv Jain graduated in 2022, he has taken SoundWatch with him to the University of Michigan and is now the PI of the project as a Professor in CSE.

Publications

Sound Sensing and Feedback Techniques for Deaf and Hard of Hearing People

Dhruv Jain

UW CS PhD Dissertation 2022

SoundWatch: deep learning for sound accessibility on smartwatches

Dhruv Jain, Hung Ngo, Pratyush Patel, Steven Goodman, Khoa Nguyen, Rachel Grossman-Kahn, Leah Findlater, Jon E. Froehlich

CACM 2022

Toward User-Driven Sound Recognizer Personalization with People who are d/Deaf or Hard of Hearing

Steven Goodman, Ping Liu, Dhruv Jain, Emma McDonnell, Jon E. Froehlich, Leah Findlater

IMWUT 2021

SoundWatch: Exploring Smartwatch-based Deep Learning Approaches to Support Sound Awareness for Deaf and Hard of Hearing Users

Dhruv Jain, Hung Ngo, Pratyush Patel, Steven Goodman, Leah Findlater, Jon E. Froehlich

Proceedings of ASSETS 2020 | Acceptance Rate: 27.5% (46 / 167) | Best Artifact Award

Field Study of a Tactile Sound Awareness Device for Deaf and Hard of Hearing Users

Dhruv Jain, Brendon Chiu, Steven Goodman, Chris Schmandt, Leah Findlater, Jon E. Froehlich

Proceedings of ISWC 2020

Evaluating Smartwatch-based Sound Feedback for Deaf and Hard-of-hearing Users Across Contexts

Steven Goodman, Susanne Kirchner, Rose Guttman, Dhruv Jain, Jon E. Froehlich, Leah Findlater

Proceedings of CHI 2020 | Acceptance Rate: 24.3% (760 / 3126)