How two Northwestern doctoral students are helping you "feel" what's on your touchscreen

[ibimage==44138==Original==none==self==ibimage_align-center]

Your smartphone is ignoring you. Well, not exactly. But, according to Joe Mullenbach and Craig Schultz, it's ignoring an integral part of you: your sense of touch.

Doctoral students at Northwestern University, Mullenbach and Schultz are spearheading the TPad Tablet Project, a research initiative devoted to outfitting mobile devices with technology that allows users to “feel” what they see on their screens.

Under the mentorship of professors J. Edward Colgate and Michael Peshkin, Mullenbach and Schultz have developed the TPad (which stands for Tactile Pattern Display), a glass sheet that lays atop a touchscreen, surrounded by a circuit board. The sheet — which has been designed for a smartwatch, the Kindle Fire (TPad Tablet) and, most recently, for the Motorola Moto G phone (Tpad Phone) — transmits ultrasonic waves to the tablet based on the image displayed. The sound waves then either increase or reduce friction, causing the device to emit “slippery” vibratory sensations in response to the pressure of touch.

Touchscreen haptics offer two fundamental advantages, Mullenbach said: luxury and accessibility.

First, haptic touchscreens offer a more sophisticated tactile experience. “It simply feels good. It's the same reason you buy a nice bottle of wine or go to a live concert. They are all high-quality sensory experiences,” he said.

Second, they more accurately reflect the way our brains process and interact with three-dimensional objects. “People completely underestimate how much they depend on their sense of touch,” he said. “Imagine if you had to rely on vision to know what you were touching with your hand. You could probably only use one hand at a time, you would knock things off your desk, you would be terribly clumsy, and it would be incredibly frustrating for you.”

[ibimage==44139==Original==none==self==ibimage_align-center]

Mullenbach holds that haptics are an inevitable, if overdue, feature in mobile devices. Until they're fully implemented, he maintains, mainstream technology will remain immature, whether consumers realize it or not.

“It's really quite obvious that we're supposed to feel [what’s on our touchscreens], just like it was obvious in silent films that you were supposed to hear them talking,” he said. “In five years you'll talk about touchscreens without haptics like we talk about old movies without sound. At the time, people are satisfied with what there is, but once you experience that missing sense, you can never go back.”

Mullenbach and Schultz have produced 15 TPad Tablets by hand, which have sold for $2,000 each to “big-name consumer electronics companies.” They’re registered as products of Tangible Haptics, a company founded by Colgate and Peshkin.

In theory, TPad technology can be used on any touchscreen, Mullenbach said, and can be applied for large-scale uses, such as industrial control, medical imaging, and car dashboards. He’s considering leveraging TPad technology into a new haptic version of Braille – an antidote to the current unwieldy, inaccessible mobile alternatives for the visually impaired.

“There is still huge smartphone adoption among the visually impaired because it's the best option out there...I spent a day using my cell phone without looking at it and with Voiceover turned on. It was incredibly frustrating to say the least,” he said.

As he discovers more ways to unite technology and sensory engagement, Mullenbach grows increasingly hopeful, envisioning an industry in which sight and touch exist as equals.

“Right now we're on a mission to reach as many fingers as we can and help people realize what they've been missing all along,” he said. “The thing I look forward to most about the future is redesigning the standard touchscreen user interface with haptics in mind from the beginning. That's when it really gets fun.”