Could Your Hand Become a Touchscreen? New Technology Makes it Happen

Touchscreen technology is improving, with Microsoft researchers developing systems that turn your own body into a keypad and others developing sensors that allow the user to control their phone without removing it from their pocket or purse.

Here's a review of two Microsoft innovations involving touchscreens.

Although this technology isn't for an actual touchscreen, it's related  in that it makes it possible to turn your hand, arm, face, leg, table, wall, really any surface imaginable into a touchscreen.

"It's conceivable that anything you can do on today's mobile devices, you will be able to do on your hand using OmniTouch," said Chris Harrison, a Ph.D. student in Carnegie Mellon's Human-Computer Interaction Institute, in the press release (via Science Daily).

OmniTouch is similar to Microsoft's Kinect program -- which makes XBox interactive -- in that it uses a depth-sensing camera that allows you to control interactive applications by tapping or dragging their fingers, just as you would on a regular touchscreen.

Watch how it works:

The examples of how this technology could be used are endless, but a few include turning your hand into a keypad or a regular pad of paper into a tablet for jotting down notes without a pen.

OmniTouch includes a depth camera and laser pico-projector, which is mounted on the user's shoulder. But, according to the release, the device ultimately could be the size of a deck of cards, or even a matchbox, so that it could fit in a pocket, be easily wearable, or be integrated into future handheld devices.

OmniTouch is able to sense 3-D motion and whether something is actual "clicked" or just hovering.

Here's more on how the technology works:

Unlike OmniTouch, which bypasses the need for a screen all together, new technology in prototype form from Microsoft -- PocketTouch -- saves users from revealing their screen all together. PocketTouch would give users a discreet way to tell incoming callers they're in a meeting or at a movie, without eve removing their phone from purse or pocket.

According to Microsoft Research, the challenges were detection of touch between many types of fabric and phone orientation:

The team resolved this by using an orientation-defining unlock gesture to determine the coordinate plane, thus initializing the device for interaction. Once initialized, user orientation can be from any direction as long as it’s consistent. PocketTouch then separates purposeful finger strokes from background noise and uses them as input.

The next challenge was to process strokes to enable text recognition of characters written over the same small physical area. Happily, the problem of recognizing multistrokes as input turned out to be a matter of adapting existing solutions.

“Microsoft Windows already contains a very rich and adaptive stroke-recognition engine,” [Hrvoje] Benko says. “So if the user is sloppy with strokes—and believe me, when you're doing it through the pocket of a jacket, the results are sloppy—these systems have the language model to handle it. That made PocketTouch a lot more robust than one would expect, as you can see from the video.”

The phone was also able to sense touch between both a fleece and jacket barrier.

Bento stated that both PocketTouch and OmniTouch are part of Microsoft's larger vision for 'unconventional use' of touch technology.

[H/T Gizmodo, Gizmodo]