For immersive technologies like AR and VR to be effective, they need to understand not only about our world, but also about us. But we don’t even understand us all that well yet; as we capture more biometric information on our quest to lose ourselves in augmented and virtual realities, we’re learning more about ourselves too.
Last month we wrote about a Disney Research project which used virtual reality to study human perception. The task—catching a ball—was simple, but the reality behind how the brain and body coordinate to get a simple task done can often confound our intuition.
Now Function has teamed up with eye-tracking specialists Tobbii to equip a professional pianist with eye-tracking glasses to see what happens (see the video heading this article). Pianist Daniel Beliavsky sets about playing some complex arrangements, and the results are quite interesting.
In the video from Beliavsky’s perspective, a yellow dot shows what his eyes are fixed on. Rather than looking constantly at his hands to ensure each note is landing correctly, his eyes are darting about, often leading the hands to the next location on the keyboard, but also frequently returning to the point between his hands where he’s able to gather information about both at the same time with his peripheral vision.
This video highlights not just the potential usefulness of eye-tracking for input in immersive technologies, but also the challenges of using the raw data to understand intent. While we can use eye-tracking input for passive things like foveated rendering, using it for interaction is a much more complex problem.
Using the input from Beliavsky’s eyes alone, it may be possible to predict the next likely moves, but because of the complexity of how the brain, hands, and eyes are interacting in this case, doing so may come with extreme difficulty. If the computer had to guess what the most important thing was to Beliavsky while he was looking between his hands, the computer might guess it’s the key he was looking at. But actually Beliavsky was using a combination of muscle memory and peripheral visual queues to make his performance work, while the point at which his eyes were looking at directly (at times) was not important at all.
The post Eye-tracking Glasses Give Glimpse into Professional Pianist’s Perception appeared first on Road to VR.
from Road to VR http://ift.tt/2o02jH7
via IFTTT
No comments:
Post a Comment