This year’s NeuroGaming Conference was four times larger than last year, and four times more interesting.

By Jon Peddie

Can you control a computer, a car, your appetite, golf swing, or com­puter game with your brain?

Alright, yes maybe, yes some of the time, but not yes all the time, reliably and repeatedly. However, with each product turn and each experience, there’s enough improvement to justify more attempts. And that’s what they’ve been doing in brain research: trying and trying again, and each time the results get better.

Neuro-gaming, as envisioned by con­ference founder Zach Lynch, will be­come a recognized category that com­bines games, brain research, sensors, and new kinds of user interfaces for computers and entertainment devices.

The majority of products on display at the emerging NeuroGaming confer­ence were head-mounted electroenceph­alogram (EEG) devices, but there were also gloves, bio-feedback, and various motion sensor systems including the Leap Motion controller. A couple of ex­hibitors used Oculus Rift headsets.

At the conference, we saw and spoke with Sony, Oculus/Facebook, Leap Mo­tion, and a half dozen or more smaller companies. Microsoft and Intel were MIA at this emerging conference, and they shouldn’t have been. The feeling at the conference was one of exuber­ance. The conference drew more than 550 very smart people to the City View room on the top floor of San Francisco’s Metreon.

Our first contact with neuro-gaming was in 2007 at CITA, where NeuroSky, founded in 2004, showed their EEG head gear. They were the pioneers rushing headlong into creating commer­cial devices, and they have helped force the industry to evolve. The head bands are used by game developers to monitor a player’s reaction to a game. They are also used in marketing to measure reactions to a text or study group to ads.

At this year’s conference, there was an equal amount of focus on using neural con­trol techniques and sensors for mood mediation, medica­tion, training, and learning. One of the anchor exhibi­tors was Luminos­ity, the brain train­ing site. They’re in­terested in adding brain sensing to demonstrate how the brain is work­ing while people are using Luminos­ity to improve their brain function, or at least their memories.

Lightweight EEG systems dominated the show. The signal-to-noise ratio in EEG is too low, so these devices do a fast Fourier transform to tease out data. The information is augmented by sen­sors to measure heart rate. One of the uses mentioned at the conference was to test game players in the development of games.

For instance, combat teams meditate together to sync themselves prior to a mission. Special ops teams find the flow state to get maximum performance. The team’s heart rate gets in sync first, and then they try and sync their brain activ­ity. Haptic feedback is used when you are giving off stress symptoms to alert you to take a breath and calm down.

In the Sony booth, a gamer played a game on the PlayStation in the usual way, with a controller, but a sensor was mounted in front of the player for eye tracking. Richard Marks, who runs So­ny’s Magic Lab, which developed the Morpheus headset, told us that the idea of augmenting traditional tools with eye tracking has a huge impact on game play. The system can better understand where people are trying to aim, or move, or what they’re looking at, and it can use that information to improve game per­formance. Marks says it makes people feel like they are better at the game, and as a result they have much more fun.

The conference covered a lot of areas with fascinating and well-known speak­ers. The AR and VR sessions were packed, and no virtual reality confer­ence these days would be complete without the presence of Palmer Luckey from Oculus Rift. He’s taking the role of Jaron Lanier, who presided over the first wave of VR enthusiasm. At this confer­ence, Luckey seemed very aware of the danger of over-hyping technology. He admitted that he wasn’t totally sure how VR would be used by everyone. He did express enthusiasm for the potential

Luckey said that VR has been visual, but he said that is only one part of the experience for people. VR will expand to include touch and it could become a peripheral to all of our sensory points. “If we can get VR right, we don’t have to do anything else,” he said. Luckey was praised elaborately by Amir Ruben, of Sixense, who has been in VR since 1990. Ruben said that Luckey and the affiliation with Facebook has brought VR to the forefront in consumer con­sciousness, and that will help move the VR industry forward. Luckey said that the Facebook deal makes the reality of VR possible. Facebook has huge re­sources, said Luckey, and that’s needed because VR can’t be done cheaply. It would have taken them an ungodly amount of time and money to get VR moving. He said it’ll be five years before Oculus makes any money and that the Facebook logo will not appear on the headset.

Arye Barnehama, co-founder of Melon, which offers a Bluetooth-en­abled headband called Brain Hat, de­lighted and challenged the audience with the idea that not only could a headband measure brainwaves, but it also could be used to stimulate them.

The Melon headband has three elec­trodes. The primary electrode is on the forehead region known as FP1, where Melon can monitor brainwave activ­ity from the prefrontal cortex similar to the NeuroSky Mindwave (and the Zeo sleep headband). Barnehama said that the Brain Hat is being used in vet­erans’ hospitals to help patients with motor control and other disabilities through Transcranial Magnetic Stimula­tion (TMS) of the brain. TMS has been used as a treatment option for drug-resistant major depressive disorder, neu­ropathic chronic pain, bi-polar disorder, and schizophrenia and is being inves­tigated for conditions such as Parkin­son’s, PTSD, and as an aid in quitting smoking.

Barnehama also spoke about group gaming to improve cognitive skills peo­ple over 65 years old. She said patients are asking for it because it’s a new way to be treated. VR, of course, is not new, and NASA and VPL had been using the wide-angle LEEP viewing lenses in various head-mounted display projects since 1985.

Sony demonstrated VR 14 years ago. Marx became fascinated with VR back then. He recognized the importance of graphics, but he also saw that the field was crowded, so he focused on input instead. As mentioned earlier, one of his interests is eye tracking.

Augmented Reality

Augmented Reality (AR) was the other big discussion, and the applications discussed weren’t games. One panel member and Tish Shute, the director of product experi­ence at Syntertainment (the company co-founded by Will Wright, creator of SimCity), defined AR as who you are, what you are, and what you’re doing—a rather original definition.

Rick Johnson of CastAR pointed out that, thanks to the smartphone market’s explosion, really sophisticated parts are now readily available at low cost. John­son added that AR could help stroke victims recover or cope better, because in AR they can act in a virtual world where physics can be altered.

Brian Mullins, CEO and founder of Daqri, an AR developer, spoke about a recent project they did for a ship­building company to teach factory-floor workers how to build ship parts. Later, he told me about how they are using AR to train welders—just another mask, but no dangerous flames; the trainee holds a stick in lieu of a torch, and the AR makes it look like a torch, as well as superimposing the weld. He said more than $100 million is spent each year on AR of that type because there is a clear cost benefit. Wearables, he speculated, will bring AR to the consumer.

The mention of wearables got Shute excited, and she encouraged the audi­ence to read Vernor Vinge’s Rainbows End. And that, in turn, prompted Mul­lins to point out that AR augments all of our senses, of which he said there are 14 or more. He used an example one of them being our ability to always sense (know) where our limbs are without looking at them.

Several speakers said the challenge of neuro-gaming is that you can’t in­troduce a single piece of neuro-gaming; a whole system has to be built or de­veloped, much like AR and VR. So, it’s going to require a lot of development to get neuro-gaming off the ground, but it was clear from the conference and the attendees that there are a lot people committed to doing just that.