Overinterpreting Computational Models of Decision-Making

Bell (1985)

Can a set of equations predict and quantify complex emotions resulting from financial decisions made in an uncertain environment? An influential paper by David E. Bell considered the implications of disappointment, a psychological reaction caused by comparing an actual outcome to a more optimistic expected outcome, as in playing the lottery. Equations for regret, disappointment, elation, and satisfaction have been incorporated into economic models of financial decision-making (e.g., variants of prospect theory).

Financial choices comprise one critical aspect of decision-making in our daily lives. There are so many choices we make every day, from the proverbial option paralysis in the cereal aisle...

...to decisions about who to date, where to go on vacation, whether one should take a new job, change fields, start a business, move to a new city, get married, get divorced, have children (or not).

And who to trust. Futuristic scenario below...

Decision to Trust

I just met someone at a pivotal meeting of the Dryden Commission. We chatted beforehand and discovered we had some common ground. Plus he's brilliant, charming and witty.

“Are you looking for an ally?” he asked. 

Neil, Laura and Stanley in Season 3 of Humans

 

Should I trust this person and go out to dinner with him? Time to ask my assistant Stanley, the orange-eyed (servile) Synthetic, an anthropomorphic robot with superior strength and computational abilities.


Laura: “Stanley, was Dr. Sommer lying to me just then, about Basswood?”

Stanley, the orange-eyed Synth: “Based on initial analysis of 16 distinct physiological factors, I would rate the likelihood of deceit or misrepresentation in Dr. Sommer's response to your inquiry at... EIGHTY-FIVE PERCENT.”

The world would be easier to navigate if we could base our decisions on an abundance of data and well-tuned weighting functions accessible to the human brain. Right? Like a computational model of trust and reputation or a model of how people choose to allocate effort in social interactions. Right?

I'm out of my element here, so this will limit my understanding of these models. Which brings me to a more familiar topic: meta-commentary on interpretation (and extrapolation).

Computational Decision-Making

I'm thinking about writing a blog post entitled, "Overinterpreting Computational Models of Decision-Making". And by "overinterpreting" I mean claiming results from a binary probabilistic decision-making task have implications for individualized cognitive behavioral therapies.

— sarcastic_f (@sarcastic_f) March 14, 2021

My motivation for writing this post was annoyance. And despair. A study on probabilistic decision-making under uncertain and volatile conditions came to the conclusion that people with anxiety and depression will benefit from focusing on past successes, instead of failures. Which kinda goes without saying. The paper in eLife was far more measured and sophisticated, but the press release said:

The more chaotic things get, the harder it is for people with clinical anxiety and/or depression to make sound decisions and to learn from their mistakes. On a positive note, overly anxious and depressed people’s judgment can improve if they focus on what they get right, instead of what they get wrong...

...researchers tested the probabilistic decision-making skills of more than 300 adults, including people with major depressive disorder and generalized anxiety disorder. In probabilistic decision making, people, often without being aware of it, use the positive or negative results of their previous actions to inform their current decisions.

The unaware shall become aware. Further advice:

“When everything keeps changing rapidly, and you get a bad outcome from a decision you make, you might fixate on what you did wrong, which is often the case with clinically anxious or depressed people...”

...individualized treatments, such as cognitive behavior therapy, could improve both decision-making skills and confidence by focusing on past successes, instead of failures...

 

The final statement on individualized CBT could very well be true, but it has nothing to do with the outcome of the study (Gagne et al., 2021), wherein participants chose between two shapes associated with differential probabilities of receiving electric shock (Exp. 1), or financial gain or loss (Exp. 2).
 

With that out of the way, I will say the experiments and the computational modeling approach are impressive. The theme is probabilistic decision-making under uncertainty, with the added bonus of volatility in the underlying causal structure (e.g., the square is suddenly associated a higher probability of shocks). People with anxiety disorders and depression are generally intolerant of uncertainty. Learning the stimulus-outcome contingencies and then rapidly adapting to change was predictably impaired.

Does this general finding differ for learning under reward vs. punishment? For anxiety vs. depression? In the past, depression was associated with altered learning under reward, while anxiety was associated with altered learning under punishment (including in the authors' own work). For reasons that were not entirely clear to me, the authors chose to classify symptoms using a bifactor model designed to capture “internalizing psychopathology” common to both anxiety and depression vs. symptoms that are unique to each disorder [ but see Fried (2021) ]¹

Overall, high scores on the common internalizing factor were associated with impaired adjustments to learning rate during the volatile condition, and this held whether the outcomes were shocks, financial gains, or financial losses. Meanwhile, high scores on anxiety-unique or depression-unique symptoms did not show this relationship. This was determined by computational modeling of task performance, using a hierarchical Bayesian framework to identify the model that best described the participants' behavior:

We fitted participants’ choice behavior using alternate versions of simple reinforcement learning models. We focused on models that were parameterized in a sufficiently flexible manner to capture differences in behavior between experimental conditions (block type: volatile versus stable; task version: reward gain versus aversive) and differences in learning from better or worse than expected outcomes. We used a hierarchical Bayesian approach to estimate distributions over model parameters at an individual- and population-level with the latter capturing variation as a function of general, anxiety-specific, and depression-specific internalizing symptoms. 

We've been living in a very uncertain world for more than a year now, often in a state of loneliness and isolation. Some of us have experienced loss after loss, deteriorating mental health, lack of motivation, lack of purpose, and difficulty making decisions. My snappish response to the press release concerns whether we can prescribe individualized therapies based on the differences between the yellow arrows on the left (“resilient people”) compared to the right (“internalizing people” — i.e., the anxious and depressed), given that the participants may not even realize they're learning anything.

 Footnote

¹ I will leave it to Dr. Eiko Fried (2021) to explain whether we should accept (or reject) this bifactor model of “shared symptoms” vs. “unshared symptoms”.

References

Bell DE. (1985) Disappointment in decision making under uncertainty. Operations research 33(1):1-27.

Gagne C, Zika O, Dayan P, Bishop SJ. (2020). Impaired adaptation of learning to contingency volatility in internalizing psychopathology. Elife 9:e61387.

Further Reading

Fried EI. (2020). Lack of theory building and testing impedes progress in the factor and network literature. Psychological Inquiry 31(4):271-88.

Guest O, Martin AE. (2021) How computational modeling can force theory building in psychological science. Perspect Psychol Sci. Jan 22:1745691620970585.

van Rooij I, Baggio G. (2021). Theory before the test: How to build high-verisimilitude explanatory theories in psychological science. Perspect Psychol Sci. Jan 6:1745691620970604.