Jon E. Froehlich
Associate Professor, CS
University of Washington
I am an Associate Professor in the Paul G. Allen School of Computer Science and Engineering at the University of Washington, a Sloan Fellow, and NSF CAREER awardee. My research is in Human-Computer Interaction (HCI) with a focus on high-value social domains such as accessibility, environmental sustainability, and STEM education. I have published over 50 peer-reviewed publications; eleven have been honored with awards, including Best Papers at ACM CHI and ASSETS and a 10-Year Impact Award at UbiComp. Since becoming a professor in 2012, I have taught ~250 undergraduate students and ~150 graduate students in HCI, Design, and UbiComp courses and served as a research advisor to nine graduate students, 50+ undergraduates, and 15 high school students. From 2012-2017, I was an Assistant Professor in Computer Science at the University of Maryland.
Recent News

Aug 17, 2023 | Jon
It was a pleasure to meet Professor Youngki Lee and his students during their visit to our lab. We discussed our work in augmented reality and computer vision as well as their new approaches to edge computing and ML. Thank you for the visit!

Aug 15, 2023 | Jon
The Project Sidewalk team led a session at the 2023 PacTrans-WSDOT Summer High School Transportation Camp talking about urban design, human mobility, and disability. Students also participated in interactive activities related to sidewalk accessibility and discussed their own experiences with inaccessible transit and urban design.

Jul 28, 2023 | Jon
We had a fun visit from Josh Myers-Dean, a PhD student from the University of Colorado Boulder interested in computer vision and accessibility. Thank you Josh for visiting!

Jul 14, 2023 | Jon
It was so wonderful to finally meet our collaborator and Utah State University Professor Brent Chamberlain. Thank you Brent for stopping by with your family to Seattle. Such a joy to meet you and soak up some of your infectious energy and positivity. Professor Chamberlain is PI on our NSF Convergence grant and seemingly has infinite overlapping personal and professional interests with me! The picture below is from my "famous" green screen in my home teaching studio that shows up often in my Zoom calls (or does it?!).

Jun 07, 2023 | Jon
We had a remarkable MHCI+D Faculty Group Retreat at UW Urban Horticulture thinking and rethinking the future of HCI+D education and important topics around ethical design, environmental sustainability, and DEI. Thanks to all who attended. Grateful for the UW HCI/d community!
Projects
Publications
Implementing a Community-Based Virtual Tool To Characterize Sidewalk Accessibility in a Northern New Jersey (NJ) Town
Extended Abstract ASCIP 2023 Annual Meeting
Implementing a Community-Based Virtual Tool To Characterize Sidewalk Accessibility in a Northern New Jersey (NJ) Town
Extended Abstract CMSC 2023 Annual Meeting
PDF | Citation | Poster | Project Sidewalk
Scaling Crowd+AI Sidewalk Accessibility Assessments: Initial Experiments Examining Label Quality and Cross-city Training on Performance
Poster Proceedings of ASSETS'22 | Acceptance Rate: 58.9% (43 / 73)
PDF | doi | Citation | Project Sidewalk • Deep Learning for Sidewalk Assessment
The Future of Urban Accessibility for People with Disabilities: Data Collection, Analytics, Policy, and Tools
Extended Abstract ASSETS'22 Workshop on The Future of Urban Accessibility
PDF | doi | Citation | Project Sidewalk • Urban Accessibility Evolution • Deep Learning for Sidewalk Assessment • Accessibility-Infused Maps
A Pilot Study of Sidewalk Equity in Seattle Using Crowdsourced Sidewalk Assessment Data
UrbanAccess 2022
PDF | doi | Citation | Project Sidewalk • Sidewalk Equity
Participatory Design of Crowd+AI Tools to Map, Analyze, and Visualize Sidewalk Accessibility
Extended Abstract Proceedings of TRANSED 2022
Towards Global-Scale Crowd+AI Techniques to Map and Assess Sidewalks for People with Disabilities
CVPR2022 Workshop: Accessibility, Vision, and Autonomy (AVA)
PDF | doi | Citation | Project Sidewalk • Deep Learning for Sidewalk Assessment
SoundWatch: deep learning for sound accessibility on smartwatches
CACM
PDF | doi | Citation | Code | SoundWatch
Visualizing Urban Accessibility: Investigating Multi-Stakeholder Perspectives through a Map-based Design Probe Study
Proceedings of CHI 2022 | Acceptance Rate: 24.7% (637 / 2579)
PDF | doi | Citation | Project Sidewalk • AccessVis • Accessibility-Infused Maps
ProtoSound: A Personalized and Scalable Sound Recognition System for Deaf and Hard-of-Hearing Users
Proceedings of CHI 2022 | Acceptance Rate: 404.9% (2579 / 637)
PDF | doi | Citation | ProtoSound
The Future of Global-Scale Spatial Data Collection and Analyses on Urban (in)Accessibility for People with Disabilities
Extended Abstract Spatial Data Science Symposium 2021
PDF | Citation | Project Sidewalk • Urban Accessibility Evolution • Deep Learning for Sidewalk Assessment • AccessVis • Accessibility-Infused Maps • Sidewalk Gallery
A Preliminary Analysis of Android Educational Game Accessibility
Extended Abstract Proceedings of ASSETS 2021 | Acceptance Rate: 61.8% (55 / 89)
Experimental Crowd+AI Approaches to Track Accessibility Features in Sidewalk Intersections Over Time
Extended Abstract Proceedings of ASSETS 2021 | Acceptance Rate: 61.8% (55 / 89)
PDF | doi | Citation | Project Sidewalk • Urban Accessibility Evolution
Sidewalk Gallery: An Interactive, Filterable Image Gallery of Over 500,000 Sidewalk Accessibility Problems
Extended Abstract Proceedings of ASSETS 2021 | Acceptance Rate: 61.8% (55 / 89) | Best Artifact Runner-up Award
PDF | doi | Citation | Project Sidewalk • Sidewalk Gallery
Toward User-Driven Sound Recognizer Personalization with People who are d/Deaf or Hard of Hearing
Proceedings of IMWUT 2021
PDF | doi | Citation | SoundWatch • Personalizing Sound Recognition
SoundWatch: Exploring Smartwatch-based Deep Learning Approaches to Support Sound Awareness for Deaf and Hard of Hearing Users
Proceedings of ASSETS 2020 | Acceptance Rate: 27.5% (46 / 167) | Best Artifact Award
PDF | doi | Citation | SoundWatch
Urban Accessibility as a Socio-Political Problem: A Multi-Stakeholder Analysis
CSCW 2020
PDF | doi | Citation | Project Sidewalk
Sidewalk Accessibility in the US and Mexico: Policies, Tools, and A Preliminary Case Study
Extended Abstract Workshop Proceedings of Civic Tech 2020
PDF | doi | Citation | Project Sidewalk
Towards Mapping and Assessing Sidewalk Accessibility Across Socio-cultural and Geographic Contexts
Extended Abstract Data4Good
PDF | doi | Citation | Project Sidewalk
Field Study of a Tactile Sound Awareness Device for Deaf and Hard of Hearing Users
Proceedings of ISWC 2020
PDF | doi | Citation | SoundWatch
Evaluating Smartwatch-based Sound Feedback for Deaf and Hard-of-hearing Users Across Contexts
Proceedings of CHI 2020 | Acceptance Rate: 24.3% (760 / 3126)
PDF | doi | Citation | SoundWatch
Deep Learning for Automatically Detecting Sidewalk Accessibility Problems Using Streetscape Imagery
Proceedings of ASSETS 2019 | Acceptance Rate: 25.9% (41 / 158) | Best Paper Award
PDF | doi | Citation | Code | Project Sidewalk • Deep Learning for Sidewalk Assessment
Autoethnography of a Hard of Hearing Traveler
Proceedings of ASSETS 2019 | Acceptance Rate: 25.9% (41 / 158) | Best Paper Nominee
PrototypAR: Prototyping and Simulating Complex Systems with Paper Craft and Augmented Reality
Proceedings of IDC 2019
PDF | doi | Citation | PrototypAR
Social Tensions with Head-Mounted Displays for Accessibility
Extended Abstract CHI2019 Workshop Proceedings of SHMD
PDF | Citation | AR Captioning
Thermporal: An Easy-to-Deploy Temporal Thermographic Sensor System to Support Residential Energy Audits
Proceedings of CHI 2019 | Acceptance Rate: 23.8% (705 / 2960)
PDF | doi | Citation | Talk | Pervasive Thermography • Thermporal
Anchored Audio Sampling: A Seamless Method for Exploring Children's Thoughts During Deployment Studies
Proceedings of CHI 2019 | Acceptance Rate: 23.8% (705 / 2960) | Best Paper Award
PDF | doi | Citation | Code | Anchored Audio Sampling
Project Sidewalk: A Web-based Crowdsourcing Tool for Collecting Sidewalk Accessibility Data at Scale
Proceedings of CHI 2019 | Acceptance Rate: 23.8% (705 / 2960) | Best Paper Award
PDF | doi | Citation | Code | Project Sidewalk
Grand challenges in accessible maps
Interactions
PDF | doi | Citation | Project Sidewalk • Accessibility-Infused Maps
Applying Transfer Learning to Recognize Clothing Patterns Using a Finger-Mounted Camera
Extended Abstract Proceedings of ASSETS 2018 | Acceptance Rate: 58.3% (42 / 72)
PDF | doi | Citation | HandSight • Cloth Recognition
Interactively Modeling and Visualizing Neighborhood Accessibility at Scale: An Initial Study of Washington DC
Extended Abstract Proceedings of ASSETS 2018
PDF | doi | Citation | Project Sidewalk • Accessibility-Infused Maps • AccessVis
A Feasibility Study of Using Google Street View and Computer Vision to Track the Evolution of Urban Accessibility
Extended Abstract Proceedings of ASSETS 2018
PDF | doi | Citation | Project Sidewalk • Urban Accessibility Evolution
Towards Accessible Conversations in a Mobile Context for People who are Deaf and Hard of Hearing
Proceedings of ASSETS 2018 | Acceptance Rate: 25.9% (28 / 108)
PDF | doi | Citation | Talk | AR Captioning
A Large-Scale Analysis of YouTube Videos Depicting Everyday Thermal Camera Use
Proceedings of MobileHCI2018
PDF | doi | Citation | Talk | Pervasive Thermography
Scaffolding Authentic Wearable-Based Scientific Inquiry for Early Elementary Learners
Proceedings of ICLS2018
PDF | Citation | Talk | BodyVis • SharedPhys
Physiological Investigations with Live Physiological Sensing and Visualization Tools
Extended Abstract Symposium Proceedings of ICLS2018
PDF | doi | Citation | BodyVis • SharedPhys
Exploring Augmented Reality Approaches to Real-Time Captioning: A Preliminary Autoethnographic Study
Extended Abstract Proceedings of DIS2018
PDF | doi | Citation | AR Captioning
SIG: Making Maps Accessible and Putting Accessibility in Maps
Extended Abstract Proceedings of CHI 2018
PDF | doi | Citation | Project Sidewalk • Accessibility-Infused Maps
Prototyping and Simulating Complex Systems with Paper Craft and Augmented Reality: An Initial Investigation
Extended Abstract Proceedings of TEI2018
PDF | doi | Citation | PrototypAR
TacTILE: A Preliminary Toolchain for Creating Accessible Graphics with 3D-Printed Overlays and Auditory Annotations
Extended Abstract Proceedings of ASSETS 2017
PDF | doi | Citation | Touchscreen Accessibility • TacTILE
Recognizing Clothing Colors and Visual Textures Using a Finger-Mounted Camera: An Initial Investigation
Extended Abstract Proceedings of ASSETS 2017
PDF | doi | Citation | HandSight • Cloth Recognition
Evaluating Wrist-Based Haptic Feedback for Non-Visual Target Finding and Path Tracing on a 2D Surface
Proceedings of ASSETS 2017 | Acceptance Rate: 26.2% (33 / 126)
PDF | doi | Citation | Touchscreen Accessibility • HandSight • Haptic Hand Guidance
A Pilot Deployment of an Online Tool for Large-Scale Virtual Auditing of Urban Accessibility
Extended Abstract Proceedings of ASSETS 2017
PDF | doi | Citation | Project Sidewalk
A Temporal Thermography System for Supporting Longitudinal Building Energy Audits
Extended Abstract Proceedings of Ubicomp 2017
PDF | doi | Citation | Pervasive Thermography • Thermporal
Exploring Novice Approaches to Smartphone-based Thermographic Energy Auditing: A Field Study
Proceedings of CHI 2017 | Acceptance Rate: 25.0% (606 / 2424)
PDF | doi | Citation | Talk | Pervasive Thermography
Comparing Touchscreen and Mouse Input Performance by People With and Without Upper Body Motor Impairments
Proceedings of CHI2017 | Acceptance Rate: 25.0% (606 / 2424)
SqueezaPulse: Adding Interactive Input to Fabricated Objects Using Corrugated Tubes and Air Pulses
Proceedings of TEI'17
PDF | doi | Citation | SqueezaPulse
Localization of Skin Features on the Hand and Wrist From Small Image Patches
Proceedings of ICPR 2016
PDF | doi | Citation | HandSight • On-Body Interaction
Evaluating Haptic and Auditory Directional Guidance to Assist Blind People in Reading Printed Text Using Finger-Mounted Cameras
ACM Transactions on Accessible Computing (TACCESS) 2016
PDF | doi | Citation | HandSight • Haptic Hand Guidance
“That's Your Heart!”: Live Physiological Sensing & Visualization Tools for Life-Relevant & Collaborative STEM Learning
Proceedings of ICLS 2016
PDF | doi | Citation | BodyVis • SharedPhys
SharedPhys: Live Physiological Sensing, Whole-Body Interaction, and Large-Screen Visualizations to Support Shared Inquiry Experiences
Proceedings of IDC 2016
PDF | doi | Citation | SharedPhys • BodyVis
Evaluating Angular Accuracy of Wrist-based Haptic Directional Guidance for Hand Movement
Proceedings of GI 2016 | Acceptance Rate: 39.4% (13 / 33)
PDF | doi | Citation | HandSight • Haptic Hand Guidance
The Future Role of Thermography in Human-Building Interaction
CHI 2016 Workshop: Future of Human-Building Interaction
The Design of Assistive Location-based Technologies for People with Ambulatory Disabilities: A Formative Study
Proceedings of CHI 2016 | Acceptance Rate: 25.0% (600 / 2400)
PDF | doi | Citation | Accessibility-Infused Maps • Project Sidewalk
Characterizing and Visualizing Physical World Accessibility at Scale Using Crowdsourcing, Computer Vision, and Machine Learning
SIGACCESS Newsletter '15
PDF | doi | Citation | Project Sidewalk
AtmoSPHERE: Representing Space and Movement Using Sand Traces in an Interactive Zen Garden
CHI '15 Extended Abstracts
PDF | doi | Citation | Interactive Sand Table
"I Like This Shirt": Exploring the Translation of Social Mechanisms in the Virtual World into Physical Experiences
CHI '15 Extended Abstracts
PDF | doi | Citation | I Like This Shirt
Understanding the Role of Thermography in Energy Auditing: Current Practices and the Potential for Automated Solutions
Proceedings of CHI 2015 | Honorable Mention
PDF | doi | Citation | Pervasive Thermography
Improving Public Transit Accessibility for Blind Riders by Crowdsourcing Bus Stop Landmark Locations with Google Street View: An Extended Analysis
TACCESS 2015
Tohme: Detecting Curb Ramps in Google Street View Using Crowdsourcing, Computer Vision, and Machine Learning
Proceedings of UIST 2014
PDF | doi | Citation | Project Sidewalk • Tohme
Towards Automated Thermal Profiling of Buildings at Scale Using Unmanned Aerial Vehicles and 3D-Reconstruction
Poster Proceedings of UbiComp 2014
PDF | doi | Citation | Pervasive Thermography
Social Fabric Fitness: The Design and Evaluation of Wearable E-Textile Displays to Support Group Running
Proceedings of CHI 2014
PDF | doi | Citation | Social Fabric Fitness
An Initial Study of Automatic Curb Ramp Detection with Crowdsourced Verification using Google Street View Images
Poster Proceedings of HCOMP 2013
PDF | doi | Citation | Project Sidewalk • Tohme
Surveying the Accessibility of Touchscreen Games for Persons with Motor Impairments: A Preliminary Analysis
Poster Proceedings of ASSETS 2013
Improving Public Transit Accessibility for Blind Riders by Crowdsourcing Bus Stop Landmark Locations with Google Street View
Proceedings of ASSETS 2013 | Best Paper Award
PDF | doi | Citation | Talk | Crowdsourcing Bus Stop Landmarks
Exploring Early Solutions for Automatically Identifying Inaccessible Sidewalks in the Physical World using Google Street View
HCIC2013 Workshop
PDF | Citation | Project Sidewalk • Tohme
Age-Related Differences in Performance with Touchscreens Compared to Traditional Mouse Input
Proceedings of CHI 2013 | Honorable Mention
Combining Crowdsourcing and Google Street View to Identify Street-level Accessibility Problems
Proceedings of CHI 2013
PDF | doi | Citation | Project Sidewalk
Bear-With-Me: An Embodied Prototype to Explore Tangible Two-Way Exchanges of Emotional Language
CHI '13 Extended Abstracts
PDF | doi | Citation | Bear-With-Me
Individuals Among Commuters: Building Personalised Transport Information Services from Fare Collection Systems
Pervasive and Mobile Computing (PMC) 2012
A feasibility study of crowdsourcing and google street view to determine sidewalk accessibility
Poster Proceedings of ASSETS 2012
PDF | doi | Citation | Project Sidewalk
Mining Public Transport Usage for Personalised Intelligent Transport Systems
Proceedings of ICDM 2010
Barrier pointing: using physical edges to assist target acquisition on mobile device touch screens
Proceedings of ASSETS 2007
Talks
Dec. 13, 2021 | Spatial Data Science Symposium 2021
Virtual
PDF | PPTX | Accessibility-Infused Maps | Project Sidewalk | AccessVis
Oct. 16, 2018 | CSE510 Advanced Topics in HCI Guest Lecture
University of Washington, Seattle
Oct. 9, 2018 | Change Seminar
University of Washington, Seattle
Oct. 9, 2018 | HCDE539: Physical Computing and Prototyping Guest Lecture
University of Washington, Seattle
PDF | PPTX | Social Fabric Fitness | I Like This Shirt | MakerWear | BodyVis
Nov. 7, 2017 | HCDE539: Physical Computing and Prototyping Guest Lecture
University of Washington, Seattle
PDF | PPTX | Social Fabric Fitness | MakerWear | BodyVis
Oct. 13, 2017 | DUB Retreat
University of Washington, Seattle
PDF | PPTX | Project Sidewalk | MakerWear | Ondulé | BodyVis
June 7, 2017 | UMD CS Staff Talk
University of Maryland, College Park
PDF | PPTX | HandSight | Project Sidewalk | MakerWear | BodyVis
April 13, 2017 | HCDE Invited Talk
University of Washington, Seattle
PDF | PPTX | HandSight | Project Sidewalk | MakerWear | BodyVis
April 11, 2017 | UW CSE Colloquium
University of Washington, Seattle
PDF | PPTX | HandSight | Project Sidewalk | MakerWear | BodyVis
April 6, 2017 | Lecture Series at the Laboratory for Telecommunication Sciences
LTS Auditorium, College Park, MD
PDF | HandSight | Project Sidewalk | Pervasive Thermography | MakerWear | BodyVis
April 4, 2017 | STEAM Salon Lecture Series
University of Maryland, College Park
Aug. 10, 2016 | Under Armour Lighthouse Partner Meeting
College Park, Maryland
May 26, 2016 | 33rd Annual HCIL Symposium
College Park, Maryland
May 12, 2016 | CS Corporate Affiliates Day
College Park, Maryland
May 28, 2015 | 32nd Annual HCIL Symposium
College Park, Maryland
March 24, 2015 | Keynote Talk at the Symposium and Hackathon in Social Media and Interaction
Emmanuel College, Cambridge University
Oct. 29, 2014 | Human-Computer Interaction Institute (HCII) Seminar Series, Carnegie Mellon University
Pittsburgh, Pennsylvania
PDF | PPTX | SlideShare
Oct. 22, 2014 | Science and Technology Society: College Park Scholars
College Park, Maryland
PDF | PPTX | SlideShare
Oct. 16, 2014 | National Academy of Sciences: D.C. Art Science Evening Rendezvous (DASER)
Washington DC
PDF | PPTX | SlideShare
May 29, 2014 | 31st Annual HCIL Symposium
College Park, Maryland
PDF | PPTX | SlideShare
April 21, 2014 | Maker Movement in Health
Indiana University (via Skype)
March 27, 2014 | Science and Technology Society: College Park Scholars
College Park, Maryland
March 24, 2014 | Talk to CS Department External Review Committee
College Park, Maryland
Feb. 12, 2014 | Human Factors and Applied Cognition Seminar Series, George Mason University
Fairfax, Virginia
Dec. 10, 2013 | Segal Design Institute Seminar Series, Northwestern University
Evanston, Illionois
Nov. 8, 2013 | Enabled by Design-athon
Washington DC
Oct. 4, 2013 | Undergraduate Research Friday Afternoon Research Talk
College Park, MD
Sept. 12, 2013 | HCIL Brown Bag Lunch Speaker Series
College Park, MD
July 30, 2013 | Digital Societies and Social Technologies Summer Institute
College Park, MD
May 22, 2013 | 30th Annual HCIL Symposium
College Park, Maryland
PDF | PPTX | SlideShare
Dec. 5, 2012 | Spatio-temporal Data Mining for a Better Understanding of Human Mobility. The Bicycle Sharing System (BSS) Case Study
Paris, France
Nov. 29, 2012 | University of Colorado: Joint Colloquium between Computer Science and the Environmental Design Program
Boulder, Colorado
Nov. 13, 2012 | Behavior, Energy, and Climate Change 2012
Sacramento, California
PDF | PPTX | SlideShare
July 23, 2012 | SESYNC Workshop on Visualization Technologies to Support Research on Human-Environment Interactions
Annapolis, Maryland
July 3, 2012 | The Center for Language and Speech Processing, Johns Hopkins University
Baltimore, Maryland
PDF | PPTX | HydroSense
June 5, 2012 | University of Maryland, Baltimore County (UMBC)
Baltimore, Maryland
PDF | PPTX | HydroSense
May 22, 2012 | 29th Annual HCIL Symposium
College Park, Maryland
May 9, 2012 | 30th international conference on Human factors in computing systems (CHI'12)
Austin, Texas
PDF | PPTX | SlideShare | HydroSense
April 21, 2012 | 22nd Annual High School Programming Contest at the University of Maryland
College Park, Maryland
Feb. 17, 2012 | Virginia Tech, North Capital Region
Falls Church, Virginia
Sept. 7, 2011 | University of Washington Computer Science Doctoral Defense
Seattle, Washington
June 13, 2011 | 9th International Conference on Pervasive computing (Pervasive'11)
San Francisco, California
PDF | PPTX | HydroSense
May 24, 2011 | Roosevelt High School
Atlanta, Georgia
April 24, 2011 | Microsoft Research
Atlanta, Georgia
April 13, 2011 | School of Interactive Computing, Georgia Institute of Technology
Atlanta, Georgia
April 6, 2011 | Department of Computer Science, Columbia University
New York City, New York
April 4, 2011 | Department of Computer Science, University of Maryland
College Park, Maryland