Project Description

Date

2015–2018
Color identification tools do not identify visual patterns or allow users to quickly inspect multiple locations, which are both important for identifying clothing. We are exploring the use of a finger-based camera that allows users to query clothing colors and patterns by touch. We initially demonstrated the feasibility of this approach using a small, highly-controlled dataset and combining two image classification techniques commonly used for object recognition. More recently, to improve scalability and robustness, we collect a dataset of fabric images from online sources and apply transfer learning to train an end-to-end deep neural network to recognize visual patterns. This new approach achieves 92% accuracy in a general case and 97% when tuned for images from a finger-mounted camera.

Publications

Applying Transfer Learning to Recognize Clothing Patterns Using a Finger-Mounted Camera

Lee Stearns, Leah Findlater, Jon E. Froehlich

Extended Abstract Proceedings of ASSETS 2018 | Acceptance Rate: 58.3% (42 / 72)

PDF | doi | Citation | Cloth Recognition • HandSight

Recognizing Clothing Colors and Visual Textures Using a Finger-Mounted Camera: An Initial Investigation

Alexander Medeiros, Lee Stearns, Leah Findlater, Chuan Chen, Jon E. Froehlich

Extended Abstract Proceedings of ASSETS 2017

PDF | doi | Citation | Cloth Recognition • HandSight

Related Projects

Tennis ball is covered by a red dot and surrounded by four green arrows forming a crosshair.
ARTennis
2023–Present
This is an icon indicating the system diagram for LabelAid
LabelAId
2023–Present
A screenshot of BusStop CV showing automatic detections
BusStopCV
2023–Present
This is the thumbnail image for the project RASSAR
RASSAR
2022–Present
This is the thumbnail image for the project ProtoSound
ProtoSound
2020–2022

We design, build, and evaluate new interactive tools and techniques to address pressing societal challenges. Makeability refers both to how our technological innovations make new abilities possible for humans as well as our educational mission to help students gain new abilities as they learn and grow through research, invention, and human-centered design.

Recent News

April 18, 2024

Jon Gives Invited Talk at AI2

March 19, 2024

UW News: Improving Mobility Technologies for Children

Feb. 29, 2024

Froehlich Gives Invited Talk on Project Sidewalk at NSF Smart and Connected Communities PI Meeting

Links