Automatically-Triggered Brain Stimulation during Encoding Improves Verbal Recall

Fig. 4 (modified from Ezzyat et al., 2018). Stimulation targets showing numerical increase/decrease in free recall performance are shown in red/blue. Memory-enhancing sites clustered in the middle portion of the left middle temporal gyrus.


Everyone forgets. As we grow older or have a brain injury or a stroke or develop a neurodegenerative disease, we forget much more often. Is there a technological intervention that can help us remember? That is the $50 million dollar question funded by DARPA's Restoring Active Memory (RAM) Program, which has focused on intracranial electrodes implanted in epilepsy patients to monitor seizure activity.

Led by Michael Kahana's group at the University of Pennsylvania and including nine other universities, agencies, and companies, this Big Science project is trying to establish a “closed-loop” system that records brain activity and stimulates appropriate regions when a state indicative of poor memory function is detected (Ezzyat et al., 2018).

Initial “open-loop” efforts targeting medial temporal lobe memory structures (entorhinal cortex, hippocampus) were unsuccessful (Jacobs et al., 2016). In fact, direct electrical stimulation of these regions during encoding of spatial and verbal information actually impaired memory performance, unlike an initial smaller study (Suthana et al., 2012).¹

{See Bad news for DARPA's RAM program: Electrical Stimulation of Entorhinal Region Impairs Memory}


However, during the recent CNS symposium on Memory Modulation via Direct Brain Stimulation in Humans, Dr. Suthana suggested that “Stimulation of entorhinal white matter and not nearby gray matter was effective in improving hippocampal-dependent memory...” ²

{see this ScienceNews story}


Enter the Lateral Temporal Cortex

Meanwhile, the Penn group and their collaborators moved to a different target region, which was also discussed in the CNS 2018 symposium: “Closed-loop stimulation of temporal cortex rescues functional networks and improves memory” (based on Ezzyat et al., 2018).

Fig. 4 (modified from Ezzyat et al., 2018). Horizontal section. Stimulation targets showing numerical increase/decrease in free recall performance are shown in red/blue. Memory-enhancing sites clustered in the middle portion of the left middle temporal gyrus.


Twenty-five patients performed a memory task in which they were shown a list of 12 nouns, followed by a distractor task, and finally a free recall phase, where they were asked to remember as many of the words as they could. The participants went through a total of 25 rounds of this study-test procedure.

Meanwhile, the first three rounds were “record-only” sessions, where the investigators developed a classifier — a pattern of brain activity — that could predict whether or not the patient would recall the word at better than chance (AUC = 0.61, where chance =.50).” ³ The classifier relied on activity across all electrodes that were placed in an individual patient.

Memory blocks #4-25 alternated between Simulation (Stim) and No Stimulation (NoStim) lists. In Stim blocks, 0.5-2.25 mA stimulation was delivered for 500 ms when the classifier AUC predicted 0.5 recall during word presentation. In NoStim lists, stimulation was not delivered on analogous trials, and the comparison between those two conditions comprised the main contrast shown below.

Fig. 3a (modified from Ezzyat et al., 2018). Stimulation delivered to lateral temporal cortex targets increased the probability of recall compared to matched unstimulated words in the same subject (P < 0.05) and stimulation delivered to Non-lateral temporal targets in an independent group (P < 0.01).


The authors found that that lateral temporal cortex stimulation increased the relative probability of item recall by 15% (using a log-binomial model to estimate the relative change in recall probability). {But if you want to see all of the data, peruse the Appendix below. Overall recall isn't that great...}

Lateral temporal cortex (n=18) meant MTG, STG, and IFG (mostly on the left). Non-lateral temporal cortex (n=11) meant elsewhere (see Appendix below). The improvements were greatest with stimulation in the middle portion of the left middle temporal gyrus. There are many reason for poor encoding, and one could be that subjects were not paying enough attention. The authors didn't have the electrode coverage to test that explicitly. This leads me to believe that electrical stimulation was enhancing the semantic encoding of the words. The MTG is thought to be critical for semantic representations and language comprehension in general (Turken & Dronkers, 2011).

Thus, my interpretation of the results is that stimulation may have boosted semantic encoding of the words, given the nature of the stimuli (words, obviously), the left lateralization with a focus in MTG, and the lack of an encoding task. The verbal memory literature clearly demonstrates that when subjects have a deep semantic encoding task (e.g., living/non-living decision), compared to shallow orthographic (are there letters that extend above/below?) or phonological tasks, recall and recognition are improved. Which led me to ask some questions, and one of the authors kindly replied (Dan Rizzuto, personal communication). ⁴

1. Did you ever have conditions that contrasted different encoding tasks? Here I meant to ask about semantic vs orthographic encoding (because the instructions were always to “remember the words” with no specific encoding task).
 
• We studied three verbal learning tasks (uncategorized free recall, categorized free recall, paired associates learning) and one spatial navigation task during the DARPA RAM project. We were able to successfully decode recalled / non-recalled words using the same classifier across the three different verbal memory tasks, but we never got sufficient paired associates data to determine whether we could reliably increase memory performance on this task.
 
2. Did you ever test nonverbal stimuli (not nameable pictures, which have a verbal code), but visual-spatial stimuli? Here I was trying to assess the lexical-semantic nature of the effect. 
 
• With regard to the spatial navigation task, we did observe a few individual patients with LTC stimulation-related enhancement, but we haven't yet replicated the effect across the population.

Although this method may have therapeutic implications in the future, at present it is too impractical, and the gains were quite small. Nonetheless, it is an accomplished piece of work to demonstrate closed-loop memory enhancement in humans.


Footnotes

¹ Since that time, however, the UCLA group has reported that theta-burst microstimulation of....

....the right entorhinal area during learning significantly improved subsequent memory specificity for novel portraits; participants were able both to recognize previously-viewed photos and reject similar lures. These results suggest that microstimulation with physiologic level currents—a radical departure from commonly used deep brain stimulation protocols—is sufficient to modulate human behavior and provides an avenue for refined interrogation of the circuits involved in human memory.

² Unfortunately, I was running between two sessions and missed that particular talk.

³ This level of prediction is more like a proof of concept and would not be clinically acceptable at this point.

⁴ Thanks also to Youssef Ezzyat and Cory Inman, whom I met at the symposium.


References

Ezzyat Y, Wanda PA, Levy DF, Kadel A, Aka A, Pedisich I, Sperling MR, Sharan AD, Lega BC, Burks A, Gross RE, Inman CS, Jobst BC, Gorenstein MA, Davis KA, Worrell GA, Kucewicz MT, Stein JM, Gorniak R, Das SR, Rizzuto DS, Kahana MJ. (2018). Closed-loop stimulation of temporal cortex rescues functional networks and improves memory. Nat Commun. 9(1): 365.

Jacobs, J., Miller, J., Lee, S., Coffey, T., Watrous, A., Sperling, M., Sharan, A., Worrell, G., Berry, B., Lega, B., Jobst, B., Davis, K., Gross, R., Sheth, S., Ezzyat, Y., Das, S., Stein, J., Gorniak, R., Kahana, M., & Rizzuto, D. (2016). Direct Electrical Stimulation of the Human Entorhinal Region and Hippocampus Impairs Memory. Neuron 92(5): 983-990.

Suthana, N., Haneef, Z., Stern, J., Mukamel, R., Behnke, E., Knowlton, B., & Fried, I. (2012). Memory Enhancement and Deep-Brain Stimulation of the Entorhinal Area. New England Journal of Medicine 366(6): 502-510.

Titiz AS, Hill MRH, Mankin EA, M Aghajan Z, Eliashiv D, Tchemodanov N, Maoz U, Stern J, Tran ME, Schuette P, Behnke E, Suthana NA, Fried I. (2017). Theta-burstmicrostimulation in the human entorhinal area improves memory specificity. Elife Oct 24;6.

Turken AU, Dronkers NF. (2011). The neural architecture of the language comprehension network: converging evidence from lesion and connectivity analyses. Front Syst Neurosci. Feb 10;5:1.


Appendix (modified from Supplementary Table 1)  

- click on image for a larger view - 

In the table above, Stim and NoStim recall percentages are for ALL words in the blocks. But:

• Only half of the words in each Stim list were stimulated, however, so this comparison is conservative. The numbers improve slightly if you compare just the stimulated words with the matched non-stimulated words. Not all subjects exhibited a significant within-subject effect, but the effect is reliable across the population (Figure 3a)

25 Years of Cognitive Neuroscience in Boston

The 25th Annual Meeting of the Cognitive Neuroscience Society starts off with a big bang on Saturday afternoon with the Big Theory versus Big Data Debate, moderated by David Poeppel.¹


Big Theory versus Big Data: What Will Solve the Big Problems in Cognitive Neuroscience?


My non-commital answers are:

(1) Both.

(2) It depends. (on what you want to do: predict behavior² (or some mental state), explain behavior, control behavior, etc.)

Abstract: All areas of the sciences are excited about the innovative new ways in which data can be acquired and analyzed. In the neurosciences, there exists a veritable orgy of data – but is that what we need? Will the colossal datasets we now enjoy solve the questions we seek to answer, or do we need more ‘big theory’ to provide the necessary intellectual infrastructure? Four leading researchers, with expertise in neurophysiology, neuroimaging, artificial intelligence, language, and computation will debate these big questions, arguing for what steps are most likely to pay off and yield substantive new explanatory insight.

Talk 1: Eve Marder – The Important of the Small for Understanding the Big

Talk 2: Jack Gallant – Which Presents the Biggest Obstacle to Advances in Cognitive Neuroscience Today: Lack of Theory or Lack of Data?

Talk 3: Alona Fyshe – Data Driven Everything

Talk 4: Gary Marcus – Neuroscience, Deep Learning, and the Urgent Need for an Enriched Set of Computational Primitives


Levels of analysis! Marr! [Poeppel is the moderator] New new new! Transformative techniques, game-changing paradigms, groundbreaking schools of thought, and multiple theories for myriad neural circuits. There is no single computational system that can possibly explain brain function at all levels of analysis (gasp! not even the Free Energy Prinicple).³

A Q&A or panel discussion would be nice... (although not on the schedule)


This Special Symposium will be preceded by the ever-exciting Data Blitz (a series of 5 minute talks) and followed by a Keynote Address by the Godfather of Cognitive Neuroscience:

Michael Gazzaniga

The Consciousness Instinct

How do neurons turn into minds? How does physical “stuff”—atoms, molecules, chemicals, and cells—create the vivid and various alive worlds inside our heads? This problem has gnawed at us for millennia. In the last century there have been massive breakthroughs that have rewritten the science of the brain, and yet the puzzles faced by the ancient Greeks are still present. In this lecture I review the the history of human thinking about the mind/brain problem, giving a big-picture view of what science has revealed. Understanding how consciousness could emanate from a confederation of independent brain modules working together will help define the future of brain science and artificial intelligence, and close the gap between brain and mind.

Plus there is a jam packed schedule of posters, talks, and prestigious award recipients/presenters on Sunday through Tuesday. Another highlight:

Symposium 3 – The Next 25 Years of Cognitive Neuroscience: Opportunities and Challenges (Brad Postle, Chair)⁴


I belong to the school of slow blogging, so I probably won't have immediate recaps. Follow #CNS2018 and enjoy the conference!



Footnotes

¹ The passenger next to me was watching the Big Bang Theory, so yay for repetition priming.

² At multiple levels of analysis, e.g. from molecular processes to motor output and all in between. Not daunting or anything. Perhaps not even possible...

³ Although Poster A87 suggests otherwise. {I think}

⁴ Unfortunately, this conflicts with Symposium 1 — Memory Modulation via Direct Brain Stimulation in Humans — which I really want to attend as well.

Universal Linguistic Decoders are Everywhere

Pereira et al. (2018) - click image to enlarge


No, they're not. They're really not. They're “everywhere” to me, because I've been listening to Black Celebration. How did I go from “death is everywhere” to “universal linguistic decoders are everywhere”? I don't imagine this particular semantic leap has occurred to anyone before. Actually, the association travelled in the opposite direction, because the original title of this piece was Decoders Are Everywhere.¹ {I was listening to the record weeks ago, the silly title of the post reminded me of this, and the semantic association was remote.}

This is linguistic meaning in all its idiosyncratic glory, a space for infinite semantic vectors that are unexpected and novel. My rambling is also an excuse to not start out by saying, oh my god, what were you thinking with a title like, Toward a universal decoder of linguistic meaning from brain activation (Pereira et al., 2018). Does the word “toward” absolve you from what such a sage, all-knowing clustering algorithm would actually entail? And of course, “universal” implies applicability to every human language, not just English. How about, Toward a better clustering algorithm (using GloVe vectors) for inferring meaning from the distribution of voxels, as determined by an n=16 database of brain activation elicited by reading English sentences?

But it's unfair (and inaccurate) to suggest that the linguistic decoder can decipher a meandering train of thought when given a specific neural activity pattern. Therefore, I do not want to take anything away from what Pereira et al. (2018) have achieved in this paper. They say:

• “Our work goes substantially beyond prior work in three key ways. First, we develop a novel sampling procedure for selecting the training stimuli so as to cover the entire semantic space. This comprehensive sampling of possible meanings in training the decoder maximizes generalizability to potentially any new meaning.”
 
• “Second, we show that although our decoder is trained on a limited set of individual word meanings, it can robustly decode meanings of sentences represented as a simple average of the meanings of the content words. ... To our knowledge, this is the first demonstration of generalization from single-word meanings to meanings of sentences.”
 
• “Third, we test our decoder on two independent imaging datasets, in line with current emphasis in the field on robust and replicable science. The materials (constructed fully independently of each other and of the materials used in the training experiment) consist of sentences about a wide variety of topics—including abstract ones—that go well beyond those encountered in training.”

Unfortunately, it would take me days to adequately pore over the methods, and even then my understanding would be only cursory. The heavy lifting would need to be done by experts in linguistics, unsupervised learning, and neural decoding models. But until then...

Death is everywhere
There are flies on the windscreen
 For a start
 Reminding us
 We could be torn apart
Tonight

---Depeche Mode, Fly on the Windscreen

Footnote

¹ Well, they are super popular right now.


Reference

Pereira F, Lou B, Pritchett B, Ritter S, Gershman SJ, Kanwisher N, Botvinick M, Fedorenko E. (2018). Toward a universal decoder of linguistic meaning from brain activation. Nat Commun. 9(1):963.

Come here
Kiss me
Now
Come here
Kiss me
Now

---ibid