Tactile Display for the Visually Impaired

A blind user tries out the tactile display during the interview.
A blind user tries out the tactile display during the interview.

To help visually impaired persons in a society predominantly developed around sighted population, I joined the interdisciplinary team of computer scientists, engineers, and designers in Disney Research, Pittsburgh, and initiated the project of “Tactile Display for the Visually Impaired” using TeslaTouch, a technology that adds tactile sensation to touch screens based on the electrovibration principle.

Exploration

Through interviews and user study with early prototypes conducted at the Library for the Blind and Physically Handicapped in Pittsburgh, Pennsylvania, we identified several key applications that visually impaired users would benefit from.

Qualitative study: observe how blind people access visual information and interact with unknown physical object. At Library for the Blind and Physically Handicapped in Pittsburgh, Pennsylvania.
Qualitative study: observe how blind people access visual information and interact with unknown physical object. At Library for the Blind and Physically Handicapped in Pittsburgh, Pennsylvania.

Quantitative user study: learn how blind people identifying shapes with TeslaTouch technology. The results inform software design in texture rendering and kinetic feedback.
Quantitative user study: learn how blind people identifying shapes with TeslaTouch technology. The results inform software design in texture rendering and kinetic feedback.

Through quantitative and qualitative studies, we narrowed down to several key problems to solve with TeslaTouch:
/Navigate touch interface on phones, tablets, and ATM.
/Read maps and independently find way to a new location.
/Identify photos and share experience with people with normal vision.

In interviews, blind volunteers suggested applications they expect on TeslaTouch
In interviews, blind volunteers suggested applications they expect on TeslaTouch

Software Implementation

User feel tactile feedback and texture on the 3M touchpad, while looking at the visuals on display. Application is build in C++ with openFrameworks libraries and Pure Data.

Welcome page demonstrates the capabilities of TeslaTouch: various texture and dynamic force feedback
Welcome page demonstrates the capabilities of TeslaTouch: various texture and dynamic force feedback

Once unlocked, three apps are available for users to try out.
Once unlocked, three apps are available for users to try out.

Navigate tactile map.
Navigate tactile map.

Identify features of a photo.
Identify features of a photo.

Type digits by press down and sliding fingers in different directions.
Type digits by press down and sliding fingers in different directions.

Hardware Implementation

Besides iterations on software, I am also redesigning the physical device based on observation of blind users interacting with daily objects and unfamiliar hand hold device.

Standalone tactile display built with back projection
Standalone tactile display built with back projection

Tabletop tactile display design
Tabletop tactile display design

CNC machining two parts of display in aluminum and wood
CNC machining two parts of display in aluminum and wood

This assistive technology is demonstrated at ACM SIGCHI Conference, Vancouver on May 7-12, 2011.

IMG_9705
IMG_9705

Special thanks to Carnegie Library of Pittsburgh’s Library for the Blind and Physically Handicapped.

MOTOACTV

MOTOACTV and integral web portal

Fitness tracker and music player by Motorola Mobility.

Piccolo

The open-source CNC platform for DIY community.

Tactile Display for the Visually Impaired

A blind user tries out the tactile display during the interview.

A TeslaTouch tactile interface designed for visually impaired people.

Visuals

Visual material created for various clients.

SpeakerWire

A fun interactive wire sculpting device that bends wire forms based on sound input.

Alice in Wonderland

Poster with details

2D and 3D visualization of Lewis Carroll’s “Alice’s Adventures in Wonderland”.

Ivy Curtain

Virtual ivy grows on a photo of window

A Java program that grows a virtual plant towards light on a photo. The generated pattern is laser cut on fabric.

Hivebots

A workshop on complex structure that emerges from simple rules

Formula Hybrid

Bulldogs Racing Car hits the ground!

Racingcar build by Bulldogs Racing Team, Yale University

Clock

Clock Hands

A “clock” that indicates where people are.

A Duet for Bicycle

Tandem bicycles that interactively generate musical sound

Kinetics

A collection of kinetic sculpture pieces