Universal Linguistic Decoders are Everywhere

Pereira et al. (2018) - click image to enlarge


No, they're not. They're really not. They're “everywhere” to me, because I've been listening to Black Celebration. How did I go from “death is everywhere” to “universal linguistic decoders are everywhere”? I don't imagine this particular semantic leap has occurred to anyone before. Actually, the association travelled in the opposite direction, because the original title of this piece was Decoders Are Everywhere.¹ {I was listening to the record weeks ago, the silly title of the post reminded me of this, and the semantic association was remote.}

This is linguistic meaning in all its idiosyncratic glory, a space for infinite semantic vectors that are unexpected and novel. My rambling is also an excuse to not start out by saying, oh my god, what were you thinking with a title like, Toward a universal decoder of linguistic meaning from brain activation (Pereira et al., 2018). Does the word “toward” absolve you from what such a sage, all-knowing clustering algorithm would actually entail? And of course, “universal” implies applicability to every human language, not just English. How about, Toward a better clustering algorithm (using GloVe vectors) for inferring meaning from the distribution of voxels, as determined by an n=16 database of brain activation elicited by reading English sentences?

But it's unfair (and inaccurate) to suggest that the linguistic decoder can decipher a meandering train of thought when given a specific neural activity pattern. Therefore, I do not want to take anything away from what Pereira et al. (2018) have achieved in this paper. They say:

• “Our work goes substantially beyond prior work in three key ways. First, we develop a novel sampling procedure for selecting the training stimuli so as to cover the entire semantic space. This comprehensive sampling of possible meanings in training the decoder maximizes generalizability to potentially any new meaning.”
 
• “Second, we show that although our decoder is trained on a limited set of individual word meanings, it can robustly decode meanings of sentences represented as a simple average of the meanings of the content words. ... To our knowledge, this is the first demonstration of generalization from single-word meanings to meanings of sentences.”
 
• “Third, we test our decoder on two independent imaging datasets, in line with current emphasis in the field on robust and replicable science. The materials (constructed fully independently of each other and of the materials used in the training experiment) consist of sentences about a wide variety of topics—including abstract ones—that go well beyond those encountered in training.”

Unfortunately, it would take me days to adequately pore over the methods, and even then my understanding would be only cursory. The heavy lifting would need to be done by experts in linguistics, unsupervised learning, and neural decoding models. But until then...

Death is everywhere
There are flies on the windscreen
 For a start
 Reminding us
 We could be torn apart
Tonight

---Depeche Mode, Fly on the Windscreen

Footnote

¹ Well, they are super popular right now.


Reference

Pereira F, Lou B, Pritchett B, Ritter S, Gershman SJ, Kanwisher N, Botvinick M, Fedorenko E. (2018). Toward a universal decoder of linguistic meaning from brain activation. Nat Commun. 9(1):963.

Come here
Kiss me
Now
Come here
Kiss me
Now

---ibid