Why your finger will never replace your mouse; It’s hard to put a finger on what’s amiss with the new gesture-based computing peripheral from Leap Motion

Why your finger will never replace your mouse

August 6, 2013: 5:00 AM ET

It’s hard to put a finger on what’s amiss with the new gesture-based computing peripheral from Leap Motion.

By John Patrick Pullen

FORTUNE — He shot me. After a good eight seconds of flailing, grabbing, and poking at the air above my desk, Frank Welty finally unholstered his sidearm and put me out of my misery. Alas, it was only a game, but I never really stood a chance. My shooter, which in this case was my pointer finger, hadn’t hit a damn thing all day. The game, Fast Iron, is just one of dozens of apps available for the newly launched Leap Motion Controller. A peripheral that lets users control their computer through hand gestures, this device showed plenty of promise when it was announced in May 2012. Now, more than a year later, the $79 product has come to market, and after a week of feeling it out, it’s hard to point a finger at what exactly is wrong with gesture-based computing, at least in its current state.Setting up the Leap Motion Controller was unexpectedly easy. I had imagined having to input measurements like the controller’s distance to the screen, or heeding requirements like keeping the device at a certain height, but none of that was necessary. Other than downloading a software suite, the peripheral was more-or-less plug-and-play, the unit powered and connected to my Mac via a USB cable.

The Leap Motion software involved minimal hand-holding, and gave the impression that gesture-based controls would be easy to master, with the sensor seeming to pick up each finger and hand rotation cleanly. But the program’s lack of finish caught my eye — running on my Mac, the software’s full-screen capability resized all my other applications windows (a huge annoyance), and its graphics looked choppy, almost like they were low-resolution. The device couldn’t detect my hands shaking with worry over these concerns, but in hindsight, they were clear indicators of a lack of finish that would plague my experience with the controller.

For example, as I continued pawing through the device’s compatible software, poor graphics soon became the least of Leap Motion’s problems. The company’s Airspace Store, a proprietary app marketplace that sells third-party created software for the device, offers users a mix of free and paid apps for both Macs and PCs. The selection is mostly games, but there are learning and productivity titles already in the mix that show the range of what the Leap Motion Controller can do.

The first app I tried out was Swoosh. Upon launching the program, my computer totally freaked out, as if I was using the Leap to shadow box. At the same time, iTunes mysteriously launched and started playing the first movie file in my library (Anchorman, if you must know), blasting Swoosh’s music over the video. This happened every time I launched the third-party title, until finally I figured out the problem. Swoosh is a deejaying app that plays music from your iTunes library, and users can “spin” and “scratch” the tracks like a record with the Leap Motion Controller. I only discovered this after locating the app’s missing full-screen window, which for some unknown reason had been minimized. And when I finally loaded a song from my library into the software, the controls were unresponsive and unpredictable. I couldn’t do much of anything with it, except for throw my hands up and walk away.

That experience became a familiar one as I continued testing the device. I downloaded Frog Dissection, and managed to somehow zip open a virtual reptile using “wax on, wax off” gestures à la Karate Kid. I fiddled with Flocking, a visualization app that looks great but bores quickly, with simulated fish swimming to bits of light controlled by my fingers. I pinched and zoomed through the virtual universe with Exoplanet but had a heck of a time touching the stars. (Imagine trying to poke a pin with a wet noodle — it will make you reconsider any belief that God sticks his fingers in our tiny earthly concerns.). Meanwhile, another visualization app, Gravilux, worked as well as it does on other platforms (Android, iOS, Mac, and PC). Its developer, Scott Snibbe, also ported his iOS-based music app, OscilloScoop, to the Leap platform, proving perhaps that the controller’s problems are more about software execution than anything else.

While the Leap Motion’s games are underwhelming, there’s no denying that entertainment plays an important role in emerging technology. Just like computer gamers from the 1980s and 1990s evolved into the computer-savvy workforce at the 2000s tech boom, and the social media generation has matured into a crop of info-fueled workers today, forward-leaning products like the Leap Motion Controller have the potential to breed a future wave of dextrous, gesture hackers.

And though it is nowhere near as powerful (or expensive) as Microsoft’s (MSFT) Kinect sensor, the Leap Motion Controller did detect every movement — to a fault. If I pointed my finger out, the device often detected my thumb, too. If I was wearing my Nike+ (NKE) Fuelband, it would think the bracelet’s shiny metal clasp was another, mystery finger. Still, as an everyday Mac user, I am jealous of the Windows 8 touch interface, and I had hoped to use the Leap Motion as a smear-free touchscreen to drive my Apple (AAPL) computer — if only I could adjust to its overactive sensors.

To run the Mac operating system, Airspace offers two apps: Touchless for Mac (they have a PC equivalent, too) and Better Touch Tool. Both free programs, Touchless was made by Leap Motion, while Better Touch Tool was developed by German indie coder Andreas Hegenberg. Touchless consists of two zones, one for “hovering” (or moving the cursor) and the other for “touching” (or clicking the mouse). Using it was precise but awkward, akin to trying to use an iPhone with magnetic gloves. Sure, you can do it, but it wasn’t really meant to work like that. The Mac operating system’s buttons were too small to be pressed with a finger, let alone a quivering, airborne ghost digit that’s as fat as a hot dog. After about a half hour of trying to acclimate to the system, I finally gave the software the finger (totally undetected!) and went back to my trusty mouse.

Better Touch Tool returned much improved results but still has a way to go. By turning actual gestures (bringing both hands together like a slow clap, for example) into keyboard shortcuts (close window, sleep, etc.) at last the Leap sprang to life. I programmed a slew of moves that made sense to me, such as three finger wag left as “back” in Safari, but when it came time to use them, I swear the device wasn’t looking. Waving, I got nothing. Flicking, not a blip of recognition. Finally, I mimed choking the wretched thing, but my computer stared back, unamused.

So, I’ve decided to wave goodbye to the Leap for now. Hopefully, the next time I plug it in, I can give it a thumbs up — and maybe then, it will know exactly what I mean.

About bambooinnovator
Kee Koon Boon (“KB”) is the co-founder and director of HERO Investment Management which provides specialized fund management and investment advisory services to the ARCHEA Asia HERO Innovators Fund (www.heroinnovator.com), the only Asian SMID-cap tech-focused fund in the industry. KB is an internationally featured investor rooted in the principles of value investing for over a decade as a fund manager and analyst in the Asian capital markets who started his career at a boutique hedge fund in Singapore where he was with the firm since 2002 and was also part of the core investment committee in significantly outperforming the index in the 10-year-plus-old flagship Asian fund. He was also the portfolio manager for Asia-Pacific equities at Korea’s largest mutual fund company. Prior to setting up the H.E.R.O. Innovators Fund, KB was the Chief Investment Officer & CEO of a Singapore Registered Fund Management Company (RFMC) where he is responsible for listed Asian equity investments. KB had taught accounting at the Singapore Management University (SMU) as a faculty member and also pioneered the 15-week course on Accounting Fraud in Asia as an official module at SMU. KB remains grateful and honored to be invited by Singapore’s financial regulator Monetary Authority of Singapore (MAS) to present to their top management team about implementing a world’s first fact-based forward-looking fraud detection framework to bring about benefits for the capital markets in Singapore and for the public and investment community. KB also served the community in sharing his insights in writing articles about value investing and corporate governance in the media that include Business Times, Straits Times, Jakarta Post, Manual of Ideas, Investopedia, TedXWallStreet. He had also presented in top investment, banking and finance conferences in America, Italy, Sydney, Cape Town, HK, China. He has trained CEOs, entrepreneurs, CFOs, management executives in business strategy & business model innovation in Singapore, HK and China.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: