Neurohackers Gone Wild!

Scene from Listening, a new neuro science fiction film by writer-director Khalil Sullins.


What are some of the goals of research in human neuroscience?

• To explain how the mind works.
• To unravel the mysteries of consciousness and free will.
• To develop better treatments for mental and neurological illnesses.
• To allow paralyzed individuals to walk again.

Brain decoding experiments that use fMRI or ECoG (direct recordings of the brain in epilepsy patients) to deduce what a person is looking at or saying or thinking have become increasingly popular as well.

They're still quite limited in scope, but any study that can invoke “mind reading” or “brain-to-brain” scenarios will attract the press like moths to a flame....

For example, here's how NeuroNews site Brain Decoder covered the latest “brain-to-brain communication” stunt and the requisite sci fi predictions:

Scientists Connect 2 Brains to Play “20 Questions”

Human brains can now be linked well enough for two people to play guessing games without speaking to each other, scientists report. The researchers hooked up several pairs of people to machines that connected their brains, allowing one to deduce what was on the other's mind.
. . .

This brain-to-brain interface technology could one day allow people to empathize or see each other's perspectives more easily by sending others concepts too difficult to explain in words, [author Andrea Stocco] said.

Mind reading! Yay! But this isn't what happened. No thoughts were decoded in the making of this paper (Stocco et al., 2015).

Instead, stimulation of visual cortex did all the “talking.” Player One looked at an LED that indicated “yes” (13 Hz flashes) or “no” (12 Hz flashes). Steady-state visual evoked potentials (a type of EEG signal very common in BCI research) varied according to flicker rate, and this binary code was transmitted to a second computer, which triggered a magnetic pulse delivered to the visual cortex of Player Two if the answer was yes. The TMS pulse in turn elicited a phosphene (a brief visual percept) that indicated yes (no phosphene indicated a “no” answer).

Eventually, we see some backpedalling in the Brain Decoder article:

Ideally, brain-to-brain interfaces would one day allow one person to think about an object, say a hammer, and another to know this, along with the hammer's shape and what the first person wanted to use it for. "That would be the ideal type of complexity of information we want to achieve," Stocco said. "We don't know whether that future is possible." 

Well, um, we already have the first half of the equation to some small degree (Naselaris et al. 2015 decoded mental images of remembered scenes)...


But the Big Prize goes to.... the decoders of covert speech, or inner thoughts!! (Martin et al. 2014)

Scientists develop a brain decoder that can hear your inner thoughts

Brain decoder can eavesdrop on your inner voice



Listening to Your Thoughts

The new film Listening starts off with a riff on this work and spins into a dark and dangerous place where no thought is private. Given the preponderance of “hearing” metaphors above, it's fitting that the title is Listening, where fiction (in this case near-future science fiction) is stranger than truth. The hazard of watching a movie that depicts your field of expertise is that you nitpick every little thing (like the scalp EEG sensors that record from individual neurons). This impulse was exacerbated by a setting which is so near-future that it's present day.


From Marilyn Monroe Neurons to Carbon Nanotubes

But there were many things I did like about Listening.¹  In particular, I enjoyed the way the plot developed in the second half of the film, especially in the last 30 minutes. On the lighter side was this amusing scene of a pompous professor lecturing on the real-life finding of Marilyn Monroe neurons (Quian Quiroga et al., 2005, 2009).

Caltech Professor: “For example, the subject is asked to think about Marilyn Monroe. My study suggests not only conscious control in the hippocampus and parahippocampal cortex, when the neuron....”

Conversation between two grad students in back of class: “Hey, you hear about the new bioengineering transfer?” ...

Caltech Professor: “Mr. Thorogood, perhaps you can enlighten us all with Ryan's gossip? Or tell us what else we can conclude from this study?”

Ryan the douchy hardware guy: “We can conclude that all neurosurgeons are in love with Marilyn Monroe.”

David the thoughtful software guy: “A single neuron has not only the ability to carry complex code and abstract form but is also able to override sensory input through cognitive effort. It suggests thought is a stronger reality than the world around us.”

Caltech Professor: “Unfortunately, I think you're both correct.”


Ryan and David are grad students with Big Plans. They've set up a garage lab (with stolen computer equipment) to work on their secret EEG decoding project. Ryan the douche lets Jordan the hot bioengineering transfer into their boys' club, much to David's dismay.

Ryan: “She's assigned to Professor Hamomoto's experiment with ATP-powered cell-binding nanotube devices.” [maybe these?]

So she gets to stay in the garage. For the demonstration, Ryan sports an EEG net that looks remarkably like the ones made by EGI (shown below on the right).

Ryan reckons they'll put cell phone companies out of business with their mind reading invention, but David realizes they have a long way to go...

Jordan the hot bioengineering transfer:  “Your mind can have a dozen thoughts in a millisecond ² [really? how can you possibly assert this?] but it takes you five seconds to say 'hi sexy'?”

Ryan the douchy hardware guy: “It's not perfect.”

Jordan: “It's crap.”

.....

Jordan points out the decoding algorithm's response time is way too slow to be useful, and that recording from “a thousand neurons” ³ isn't enough... “you have to open the books.” David points out they're not neurosurgeons (who would implant intracranial electrodes for ECoG).

Jordan: “You don't need surgery... you need nanotubes.”

...and this leads to the most ridiculous scenario: intrathecal administration of said nanotubes [along with microscopic transistors to form molecular electrodes] via lumbar puncture (spinal injections) performed by complete novices wielding foot long needles. [direct administration into the cerebrospinal fluid bypasses difficulties with the impermeable blood brain barrier.] But if you can get through that, and the heavy handed use of color filters...

...you will be transported to the Red Room, where scary bald men “listen” to every thought [the direct brain-to-brain communication is one way only to avoid that nasty "circular feedback loop"].

Then more THINGS happen. It's not perfect. But it's not crap. I thought Listening was worth $4.99.

Available on Amazon and Vimeo.

Sometimes even The Neurocritic is willing to suspend disbelief...


Further Reading

Brain decoding: Reading minds: 2013 Nature News story by Kerri Smith.
“By scanning blobs of brain activity, scientists may be able to decode people's thoughts, their dreams and even their intentions.”

Neuroscience: ‘I built a brain decoder': BBC Future
“What are you looking at? Scientist Jack Gallant can find out by decoding your thoughts, as Rose Eveleth discovers.”

Brain Decoding Project: mouse hippocampus
---A BRAIN Project: Brain Activity Mapping of Neural Codes for memory

One more step along the long road towards brain-to-brain interfaces: Nice blog coverage of the 20 Questions study by Pierre Mégevand.

Meet the Hackers Who Are Decrypting Your Brainwaves: Oh no they're not. But an interesting piece on the DIY EEG movement.


Footnotes

¹ Some of the dialogue and the interpersonal relationships? Not as much.

² Dozens of thoughts in 1/1000 of a second?? Perhaps she's being hyperbolic here... Well, popular lore says we have 70,000 thoughts per day, which comes out to only 0.8101851851851852 thoughts per second. But this is also absurd, since we haven't yet defined what a “thought” even is. Interesting factoid: the Laboratory of Neuroimaging (LONI) at UCLA has taken credit for this number. But they did offer some caveats:

*This is still an open question (how many thoughts does the average human brain processes in 1 day). LONI faculty have done some very preliminary studies using undergraduate student volunteers and have estimated that one may expect around 60-70K thoughts per day. These results are not peer-reviewed/published. There is no generally accepted definition of what "thought" is or how it is created. In our study, we had assumed that a "thought" is a sporadic single-idea cognitive concept resulting from the act of thinking, or produced by spontaneous systems-level cognitive brain activations.

theoracleofdelphi-ga had some interesting thoughts on the matter:

So there's the heart of the problem: No one really knows what the biological basis for a 'thought' is, so we can't compute how fast a brain can produce them. Once you figure out the biological basis for a thought (and return from the Nobel ceremony) you can ask the question again and expect a reasonable scientific answer.

In the mean time, you could probably get a bunch of psychologists to argue about the definition of a thought for a while, and get a varying set of answers that depend highly on the definitions.

Oh, I think they said also 30 thoughts per second at another time in the movie...

³ Yeah, here's the “one electrode, one neuron” fallacy. The reality is that a single EEG electrode records summed, synchronous activity from thousands of neurons, at the very least.


References

Herff C, Heger D, de Pesters A, Telaar D, Brunner P, Schalk G, Schultz T. (2015). Brain-to-text: decoding spoken phrases from phone representations in the brain. Front Neurosci. 9:217.

Huth AG, Nishimoto S, Vu AT, Gallant JL. (2012). A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron 76(6):1210-24.

King CE, Wang PT, McCrimmon CM, Chou CC, Do AH, Nenadic Z. (2015). The feasibility of a brain-computer interface functional electrical stimulation system for the restoration of overground walking after paraplegia. J Neuroeng Rehabil. 12(1):80.

Liu H, Agam Y, Madsen JR, Kreiman G. (2009). Timing, timing, timing: fast decoding of object information from intracranial field potentials in human visual cortex. Neuron 62(2):281-90.

Martin S, Brunner P, Holdgraf C, Heinze HJ, Crone NE, Rieger J, Schalk G, Knight RT, Pasley BN.  (2014). Decoding spectrotemporal features of overt and covertspeech from the human cortex. Front Neuroeng. 7:14.

Naselaris T, Olman CA, Stansbury DE, Ugurbil K, Gallant JL. (2015). A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes. Neuroimage 105:215-28.

Pasley BN, David SV, Mesgarani N, Flinker A, Shamma SA, Crone NE, Knight RT, Chang EF. (2012). Reconstructing speech from human auditory cortex. PLoS Biol. 10(1):e1001251.

Quian Quiroga R, Kraskov A, Koch C, Fried I. (2009). Explicit encoding of multimodal percepts by single neurons in the human brain. Curr Biol. 19(15):1308-13.

Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I. (2005). Invariant visual representation by single neurons in the human brain. Nature 435(7045):1102-7.

Stocco, A., Prat, C., Losey, D., Cronin, J., Wu, J., Abernethy, J., & Rao, R. (2015). Playing 20 Questions with the Mind: Collaborative Problem Solving by Humans Using a Brain-to-Brain Interface PLOS ONE, 10 (9) DOI: 10.1371/journal.pone.0137303