HoloSound

Quick Info

Project Date Jan. 1, 2020 - Present
Sponsors: NSF Microsoft Google UW Reality Lab
Keywords: real-time captioning, sound awareness, deaf and hard of hearing

News

Dhruv Jain Receives Microsoft Dissertation Grant

Aug 18, 2021 | Jon

Congratulations to Dhruv Jain who was selected for the prestigious Microsoft Research Dissertation Grant for his work on “Sound Sensing and Feedback Techniques for Deaf and Hard of Hearing Users.” The MSR Dissertationg Grant aims to "increase the pipeline of diverse talent receiving advanced degrees in computing-related fields by providing a research funding opportunity for doctoral students who are underrepresented in the field of computing." Read more on the Allen School news page.

Dhruv Jain Selected for Google-CMD-IT LEAP Alliance Fellowship

Aug 05, 2021 | Jon

Congratulations to Dhruv Jain who was selected for the Google-CMD-IT LEAP Alliance Fellowship. CMD-IT stands for the Center for Minorities and People with Disabilities in IT and is dedicated to "promoting innovation that enriches, enhances, and enables our target communities, such that more equitable and sustainable contributions are possible by all communities."

About

HoloSound: Combining Speech and Sound Identification for Deaf or Hard of Hearing Users on a Head-mounted Display

Head-mounted displays can provide private and glanceable speech and sound feedback to deaf and hard of hearing people, yet prior systems have largely focused on speech transcription. We introduce HoloSound, a HoloLens-based augmented reality (AR) prototype that uses deep learning to classify and visualize sound identity and location in addition to providing speech transcription.