Makeability Lab
  • Home
  • News
  • People
  • Projects
  • Publications
  • Lab Handbook
  • Apply
Cloth Recognition
Cloth Recognition

Quick Info

Project Date Jan. 1, 2015 - Dec. 1, 2018
Sponsors: DoD
Keywords: handsight, computer vision, blind, cloth recognition

About

Prototype finger-worn camera to recognize clothing textures and colors by sweeping the finger across the fabric

Color identification tools do not identify visual patterns or allow users to quickly inspect multiple locations, which are both important for identifying clothing. We are exploring the use of a finger-based camera that allows users to query clothing colors and patterns by touch. We initially demonstrated the feasibility of this approach using a small, highly-controlled dataset and combining two image classification techniques commonly used for object recognition. More recently, to improve scalability and robustness, we collect a dataset of fabric images from online sources and apply transfer learning to train an end-to-end deep neural network to recognize visual patterns. This new approach achieves 92% accuracy in a general case and 97% when tuned for images from a finger-mounted camera.

Publications

Applying Transfer Learning to Recognize Clothing Patterns Using a Finger-Mounted Camera

Lee Stearns, Leah Findlater, Jon E. Froehlich

Extended Abstract Proceedings of ASSETS 2018 | Acceptance Rate: 58.3% (42 / 72)

keywords: wearables, computer vision, visual impairments, pattern recognition, transfer learning

PDF | doi | Citation | HandSight • Cloth Recognition

Recognizing Clothing Colors and Visual Textures Using a Finger-Mounted Camera: An Initial Investigation

Alexander Medeiros, Lee Stearns, Leah Findlater, Chuan Chen, Jon E. Froehlich

Extended Abstract Proceedings of ASSETS 2017

keywords: wearables, blind, visual impairments, texture recognition, cloth recognition

PDF | doi | Citation | HandSight • Cloth Recognition

We design, build, and evaluate interactive tools and techniques to address pressing societal challenges in accessibility, sustainability, education, and beyond.

Recent News

March 04, 2023

Chu Li Presents Project Sidewalk to World IA Day

March 03, 2023

Makeability Lab Members at the Annual Allen School Ski Day

March 02, 2023

Congrats to Stefania Druga on Passing the PhD Defense!

Links

  •  >   Home
  •  >   People
  •  >   News
  •  >   Publications
  •  >   Talks
  •  >   Videos