This year’s NeuroGaming Conference was four times larger than last year, and four times more interesting.
By Jon Peddie
Can you control a computer, a car, your appetite, golf swing, or computer game with your brain?
Alright, yes maybe, yes some of the time, but not yes all the time, reliably and repeatedly. However, with each product turn and each experience, there’s enough improvement to justify more attempts. And that’s what they’ve been doing in brain research: trying and trying again, and each time the results get better.
Neuro-gaming, as envisioned by conference founder Zach Lynch, will become a recognized category that combines games, brain research, sensors, and new kinds of user interfaces for computers and entertainment devices.
The majority of products on display at the emerging NeuroGaming conference were head-mounted electroencephalogram (EEG) devices, but there were also gloves, bio-feedback, and various motion sensor systems including the Leap Motion controller. A couple of exhibitors used Oculus Rift headsets.
At the conference, we saw and spoke with Sony, Oculus/Facebook, Leap Motion, and a half dozen or more smaller companies. Microsoft and Intel were MIA at this emerging conference, and they shouldn’t have been. The feeling at the conference was one of exuberance. The conference drew more than 550 very smart people to the City View room on the top floor of San Francisco’s Metreon.
Our first contact with neuro-gaming was in 2007 at CITA, where NeuroSky, founded in 2004, showed their EEG head gear. They were the pioneers rushing headlong into creating commercial devices, and they have helped force the industry to evolve. The head bands are used by game developers to monitor a player’s reaction to a game. They are also used in marketing to measure reactions to a text or study group to ads.
At this year’s conference, there was an equal amount of focus on using neural control techniques and sensors for mood mediation, medication, training, and learning. One of the anchor exhibitors was Luminosity, the brain training site. They’re interested in adding brain sensing to demonstrate how the brain is working while people are using Luminosity to improve their brain function, or at least their memories.
Lightweight EEG systems dominated the show. The signal-to-noise ratio in EEG is too low, so these devices do a fast Fourier transform to tease out data. The information is augmented by sensors to measure heart rate. One of the uses mentioned at the conference was to test game players in the development of games.
For instance, combat teams meditate together to sync themselves prior to a mission. Special ops teams find the flow state to get maximum performance. The team’s heart rate gets in sync first, and then they try and sync their brain activity. Haptic feedback is used when you are giving off stress symptoms to alert you to take a breath and calm down.
In the Sony booth, a gamer played a game on the PlayStation in the usual way, with a controller, but a sensor was mounted in front of the player for eye tracking. Richard Marks, who runs Sony’s Magic Lab, which developed the Morpheus headset, told us that the idea of augmenting traditional tools with eye tracking has a huge impact on game play. The system can better understand where people are trying to aim, or move, or what they’re looking at, and it can use that information to improve game performance. Marks says it makes people feel like they are better at the game, and as a result they have much more fun.
The conference covered a lot of areas with fascinating and well-known speakers. The AR and VR sessions were packed, and no virtual reality conference these days would be complete without the presence of Palmer Luckey from Oculus Rift. He’s taking the role of Jaron Lanier, who presided over the first wave of VR enthusiasm. At this conference, Luckey seemed very aware of the danger of over-hyping technology. He admitted that he wasn’t totally sure how VR would be used by everyone. He did express enthusiasm for the potential
Luckey said that VR has been visual, but he said that is only one part of the experience for people. VR will expand to include touch and it could become a peripheral to all of our sensory points. “If we can get VR right, we don’t have to do anything else,” he said. Luckey was praised elaborately by Amir Ruben, of Sixense, who has been in VR since 1990. Ruben said that Luckey and the affiliation with Facebook has brought VR to the forefront in consumer consciousness, and that will help move the VR industry forward. Luckey said that the Facebook deal makes the reality of VR possible. Facebook has huge resources, said Luckey, and that’s needed because VR can’t be done cheaply. It would have taken them an ungodly amount of time and money to get VR moving. He said it’ll be five years before Oculus makes any money and that the Facebook logo will not appear on the headset.
Arye Barnehama, co-founder of Melon, which offers a Bluetooth-enabled headband called Brain Hat, delighted and challenged the audience with the idea that not only could a headband measure brainwaves, but it also could be used to stimulate them.
The Melon headband has three electrodes. The primary electrode is on the forehead region known as FP1, where Melon can monitor brainwave activity from the prefrontal cortex similar to the NeuroSky Mindwave (and the Zeo sleep headband). Barnehama said that the Brain Hat is being used in veterans’ hospitals to help patients with motor control and other disabilities through Transcranial Magnetic Stimulation (TMS) of the brain. TMS has been used as a treatment option for drug-resistant major depressive disorder, neuropathic chronic pain, bi-polar disorder, and schizophrenia and is being investigated for conditions such as Parkinson’s, PTSD, and as an aid in quitting smoking.
Barnehama also spoke about group gaming to improve cognitive skills people over 65 years old. She said patients are asking for it because it’s a new way to be treated. VR, of course, is not new, and NASA and VPL had been using the wide-angle LEEP viewing lenses in various head-mounted display projects since 1985.
Sony demonstrated VR 14 years ago. Marx became fascinated with VR back then. He recognized the importance of graphics, but he also saw that the field was crowded, so he focused on input instead. As mentioned earlier, one of his interests is eye tracking.
Augmented Reality (AR) was the other big discussion, and the applications discussed weren’t games. One panel member and Tish Shute, the director of product experience at Syntertainment (the company co-founded by Will Wright, creator of SimCity), defined AR as who you are, what you are, and what you’re doing—a rather original definition.
Rick Johnson of CastAR pointed out that, thanks to the smartphone market’s explosion, really sophisticated parts are now readily available at low cost. Johnson added that AR could help stroke victims recover or cope better, because in AR they can act in a virtual world where physics can be altered.
Brian Mullins, CEO and founder of Daqri, an AR developer, spoke about a recent project they did for a shipbuilding company to teach factory-floor workers how to build ship parts. Later, he told me about how they are using AR to train welders—just another mask, but no dangerous flames; the trainee holds a stick in lieu of a torch, and the AR makes it look like a torch, as well as superimposing the weld. He said more than $100 million is spent each year on AR of that type because there is a clear cost benefit. Wearables, he speculated, will bring AR to the consumer.
The mention of wearables got Shute excited, and she encouraged the audience to read Vernor Vinge’s Rainbows End. And that, in turn, prompted Mullins to point out that AR augments all of our senses, of which he said there are 14 or more. He used an example one of them being our ability to always sense (know) where our limbs are without looking at them.
Several speakers said the challenge of neuro-gaming is that you can’t introduce a single piece of neuro-gaming; a whole system has to be built or developed, much like AR and VR. So, it’s going to require a lot of development to get neuro-gaming off the ground, but it was clear from the conference and the attendees that there are a lot people committed to doing just that.