Multi-touch Display Uses All Ten Fingers

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Multi-touch screen technologies are the latest thing for high-tech gadgets, from Apple’s iPhone to Microsoft’s Surface tabletop computer.

But to date, there has been a limit to how useful a multi-touch interface could be. It’s not practical in most circumstances, for instance, to put all ten fingers on a touch screen at once. For one thing, since the screen is there to display information, putting all of your fingers on the screen blocks the view of the information.

Now, a joint project between Microsoft Research (MSR) and Mitsubishi Electric Research Labs (MERL) has resulted in a novel solution – touch screens on both the front and back of a mobile device that are used simultaneously. The user’s thumbs touch the front screen while the other eight fingers touch the back on what’s been termed the LucidTouch “see through” mobile device interface.

To be precise, the device’s screen is not actually transparent. Instead, a camera on a short boom attached to the prototype’s back captures images of the user’s hands touching the rear screen, and an image of the fingers – to enable the user to “see” where his or her fingers are touching – is displayed on the front screen as if the device was transparent.

With the thumbs on the front screen and fingers touching the back, for example, users are able to type more easily – using a slightly reoriented QWERTY keyboard layout — than on other touch screen interfaces. The fact that users can have multiple fingers on the screen at once, without blocking (or “occluding”) the view also enables easier use of maps, for instance.

However, actually making the device transparent front to back was too problematic, according to MSR research scientist Patrick Baudisch. That’s why the researchers adopted the term “pseudo-transparent” – the transparency is actually an intentional optical illusion.

“It’s hard to get the illumination right [if it were actually transparent and] … we can stick with a regular design with the electronics in the middle of the device [instead of having to position the chips and motherboard around the edges],” Baudisch told internetnews.com.

But not showing the fingers was confusing to users, so the researchers figured out a way – using a camera at this point but later probably surface-based sensors – to show the finger locations, as if the device’s screen was actually transparent.

The current prototype uses a form factor that is similar in size to Microsoft’s “ultra-mobile PC” or UMPC design. However, the researchers continue to experiment and are looking at other form factors, such as personal digital assistants (PDA) and tablet PC devices.

Additionally, Baudisch is quick to point out that the project is a true collaborative effort with Mitsubishi researchers, including MERL’s Daniel Wigdor.

“Daniel did a lot of the engineering,” Baudisch added.

The researchers plan to publish a paper in early October when they present their work at the Association for Computing Machinery’s (ACM) Symposium on User Interface Software and Technology.

While the work is not, as such, a product design – rather it is a research project – Microsoft’s recently introduced Surface computer began in a somewhat similar manner, as an outgrowth of an MSR research project. So it is not outside the realm of possibilities that follow-on research might yield usable technologies at some future point in time.

In the meantime, however, don’t look for devices with the LucidTouch technology on the shelves at Best Buy any time soon.

“This is all just starting,” Baudisch said.

This article was first published on InternetNews.com.