In the last five years, technology has evolved exponentially— from wearables to touch screens, to motion tracking and beyond. As you might guess, the core of this evolution is driven by self-styled geeks and gurus, constantly trying to build new and better equipment to project their imagination onto the world.
Over the last decade, I would argue that gaming has driven much of this progress. Better graphics and video cards have enabled a whole new dimension of PC gaming, and even Mac has gotten on board supporting big name titles and online distribution channels. At the same time, a new generation of console games pushed the limits of game’s realism and immersive quality, while the Wii and then Xbox introduced new ways to control and play games in using motion-tracking. The big question is: what’s next? Media and film have tried to imagine the next generation of games and the generation after that time and again. However, by looking at some developments made in the last couple of weeks, we may be able to start weaving together a picture of how things will connect and extrapolate what could be possible not too far into the future.
Let’s start by jumping way down the road and talking about pretty much the ultimate place games can go, making you feel in every way like you are part of it. One recent piece of media that explores this is a great anime series MMORPGs everywhere will appreciate and laugh over, Sword Art Online (SAO). This anime takes place in 2022, so a not so distant future, when the release of a new MMORPG called Sword Art Online is released in Japan.
“Sword Art Online (SAO) is a Virtual Reality Massively Multiplayer Online Role-Playing Game (VRMMORPG), released in 2022. With the Nerve Gear, a virtual reality helmet that stimulates the user’s five senses via their brain, players can experience and control their in-game characters with their minds.”
Now, the plot of this show shifts toward the dark implications of this new technology and consequent game—the creator traps all the players inside and rigs it so that if they die in-game, the nerve gear will fry their brain and kill them in real life, and same if it is removed. But another interesting part about this narrative is how the user interface in-game plays out in a virtual world. Obviously, if you are fully immersed, you won’t have a screen to look, so where does the menu bar go? How do you pull up your inventory? These are all important questions for VR games of the future. In SAO, menus appear when hands are raised in certain positions, you can press and swipe at type “buttons” floating in mid-air, etc.
While SAO may seem like a long shot, we actually aren’t that far away from being able to functionally intercept many of the senses. We might not have five yet, but I definitely think a combination of new technology developments may plant the seeds and relationships to push us much closer to making our own SAO by 2022—or at least something that can capture senses to create a similarly immersive experience.
Want proof that video games and media can push these advances forward since people want to (and I’d say recently expect to) be able to do all the things in their favorite games/shows? Well, there are tons of videos with proof of concept, demos, and even imaginative uses for these exact menus from SAO floating around the internet.
Ultrasonic Haptic Feedback Technology Developed & Refined
This week, I ran across a great Mashable article—”Ultrasonics may soon let you touch what you see in virtual reality“—that discusses some awesome new technology developed by a UK company, UltraHaptics. Originally developed at the University of Bristol, the developers have figured out a way to provide haptic feedback for holographic imagery using ultrasonic technology.
The project’s website at the University of Bristol describes it like this:
“UltraHaptics is a system for creating haptic feedback in mid-air. Waves of ultrasound displace the air, creating a pressure difference. By causing many waves to arrive at the same place simultaneously, a noticeable pressure difference is created at that point. With this method, we are able to create multiple, concurrent points of haptic feedback in mid-air.”
So, basically, you can receive haptic or “touch” feedback from virtual images and actually sense shapes through ultrasonic waves that pulse the feedback at different strengths by changing their modulation or frequency. The resulting change in pressure against your skin that makes you feel the virtual shapes and the sensation of force is defined as “acoustic radiation pressure.”
There are some great artist renderings that Mashable featured in their article that came from the University of Bristol—although I couldn’t find them on their site.
Let’s put this into the context of gaming. Take an already developed virtual reality headset like Oculus Rift combined with a motion capture system like Microsoft’s Xbox Kinect, PlayStation 4 Camera for whole body gestures, and a technology like Leap Motion (which can sense the position of your hands and fingers and translate the motions into commands for your computer) for fine movements of the hands and fingers. What do you get? Something starting down the path to creating the experience in SAO.
In SAO, you have all five senses covered. With a combination of these technologies—and let’s throw in Omni, which lets you move around in virtual worlds, for good measure—you can get three: sight, sense, and hearing (assuming you wear a headset and microphone). Like in SAO, we might soon see hovering menus in mid-air that you can summon with a hand gesture and swipe and jab to access items and upgrades. And you’ll be able to feel yourself pushing the button. Or maybe you’ll wield a virtual sword or spear or mace and feel the pressure of the hilt in your hand. Maybe you’ll pull a virtual bow and feel the draw, hear the thrum of the string and the snap of the string on your arm. Maybe in five years or so, we will have immersive new MMORPGs that allow you to literally “plug-in” and experience gaming in a whole new way. Maybe others will take this technology and create accessories that integrate the experience even further. I know that is a lot of maybes, but who knows where this will go. Either way, it’s coming.
Just as a side note: there actually is a company working on something similar to “Nerve Gear,” which has been dubbed Brain Computer Interfaces or BCI for short. Check out this awesome company Emotiv and read about how this could work on their blog.
Facebook’s Oculus Rift Acquires NimbleVR and 13th Lab, Adding 3D Modeling and Hand-Tracking Expertise
Did I mention it’s coming? Well, it’s actually coming pretty soon. Some of the leading companies in the various tech areas I listed above—working to create an immersive in-game experience—are coming together via acquisitions and creating new synergies and relationships between its products. Last week, virtual reality company Oculus Rift, now owned by Facebook, acquired two companies with expertise in hand-tracking and 3D modeling.
Nimble VR had actually been kickstarting a campaign for a program called Sense that would work with Oculus’ headset anyway, and called it all off when it was acquired. Sense is able to create an infrared image of the world using laser technology that allows it to track the motion and position of each finger with every joint and angle. With this precise technology, it is able to allow players to control user interfaces and virtual objects without having to wear or use any objects at all. Can you say “floating mid-air menus?”
It also acquired a Swedish company, 13th Lab, which produced a technology that maps accurate 3D model of the world. It can allow someone to visit an actual 3D model of a real-world place like ancient architecture or a skyscraper. It can also be used to recognize and track 2D images. Soon we will be able to take a virtual tour of places like the Parthenon, the Vatican, and the pyramids. Or maybe we will be able to play games on top of movie models or actual models that people build. Can I download a 3D model of a norman castle and defend it from a siege, please?
On top of it all, Oculus announced late in the week that it hired motion capture guru Chris Bregler to lead a vision research team. Now it will have expertise in mapping human movements onto the digital world as well. What could they be planning … hmm?
You should check out Crystal Rift on the Rift Arcade. It is a puzzle game set in labyrinth that also reminds me of a castle. When I see this all I can think of is VR dungeon crawling!
So in just the last couple of weeks we’ve seen things coming together that can and will have huge implications for gaming when it finally makes it to commercial application. Already, developers who have their hands on Oculus’ Development Kit 2 are starting to program apps and games that take advantage of the virtual environment and will go off in all sorts of interesting directions as different types of people explore how to create and play in this new space.
It’s an exciting time to watch technology evolve and it’s an exciting time to play games. Who knows what will happen as these systems converge? It will definitely transform how we game, and maybe how we live and interact to some degree. As for me, I’m ready to throw on the VR headset, switch on all the other gadgets that let me touch and feel the virtual world around me, boot up the ol’ motion capture system to track my movements, and just plug-in. After all, hand motions and finger patterns for advanced black magic spells are pretty complicated. At least when I try it for the first time, who knows how addictive that power could be? Imagine actually feeling yourself run up a hill and sink a broadsword into the neck of some orc or some other player?
Moral implications aside, I can just imagine the excitement stirred by the boot-up and inevitable intro cutscene of some of my favorite MMORPGs, then re-imagining it through a first-player 3D lens. It’s enough to fill me with a swell of anticipatory pleasure. I can’t wait to play these games, but until then, let’s all keep our eyes on these companies, figure out what is possible, and then create them ourselves!
Finally, just to get you excited: here is a brief gameplay of Sword Art Online that is being created for Oculus Rift. Didn’t I say it was coming?