Here be values (in the brain): how the ventral striatum participates in decision-making
How do we make decisions? The question is of huge importance for every aspect of our lives. Research into the neuronal mechanisms underlying decision-making has made considerable progress in recent years, from basic research in animal models through functional neuroimaging in humans to clinical studies in patient populations. As a result, we are starting to have a reasonably complete picture of the brain areas involved in this process, which include among others the ventral striatum, a subset of the basal ganglia in the depth of each cerebral hemisphere, and the ventromedial prefrontal cortex. The former is generally linked to learning, especially habit learning, while the latter is thought to be important for the online regulation of behavior. One thing that remains unclear, however, is what the role of these individual brain areas is exactly. In a recent study published in PLOS Biology, Caleb Strait and colleagues at the University of Rochester, NY, investigated the role of the ventral striatum in making choices based on the expectation of a reward, and compared it with that of the ventromedial prefrontal cortex. Their intriguing results show that both areas reflect mostly overlapping aspects of decision-making, and suggest that choices are the result of distributed neuronal computations occurring in multiple brain regions.
Gambling monkeys seek risk
In the study, monkeys performed a gambling task, with drinking water as a reward. On each trial, they had to choose one of two options, each of which varied in the magnitude of the reward (the number of water drops) and its likeliness (say, 50% vs. 80%). The two options were presented to the monkeys one after the other, with the probability of the reward and its magnitude represented as rectangles of varying size and color respectively. Neuronal activity was recorded in the ventral striatum throughout the task. The monkeys’ choices indicated that they understood well the mechanisms of the gamble: on 83% of trials, they chose the option with the larger expected value (that is, the option that would bring the larger cumulative reward if it were chosen a large number of times). Further, the monkeys were risk-seekers: when presented with two options of equal expected value, they more often chose the riskier one (that is, the option with the smaller probability, but the larger reward amount).
Neural correlates of value in the ventral striatum
More than half the neurons recorded in the ventral striatum responded to some aspect or another of the gamble. About 15% of neurons changed their activity as a function of the reward amplitude in the first offer (that is, they fired either more or less than baseline, but consistently so for a given reward amplitude), and the same proportion responded to the probability of that first offer. Collectively, the neurons encoded both the reward amplitude and its probability within one abstract representation (that is, they tended to react in a similar fashion both when the reward probability of the first offer increased and when its magnitude increased).
When the second offer appeared on the screen, neurons in the ventral striatum responded to the magnitude and probability of the reward in a similar fashion as they did during the presentation of the first offer. Interestingly, however, a proportion of neurons also encoded the expected value of the first offer during the presentation of the second one: for instance, for a given expected value of offer 2, neurons would fire more when it was larger than that of offer 1 than when it was smaller. Thus, these neurons are encoding the comparison between the two offers by weighing their relative expected values; the authors name this antagonistic coding.
Ventral striatal neurons track eventual choice
The activity of ventral striatal neurons also tended to represent the offer that the monkey eventually chose (whether the first or the second one) more and more as each trial progressed; that effect became significant before the monkeys gave their response. Looking now at the period during which the monkeys received their reward (or not!), the authors found that about 35% of ventral striatal neurons encoded the outcome of the gamble. Importantly, neurons used the same (or at least a similar) code to represent the expected value of an offer and the actual outcome of the trial. Finally, the authors showed that neurons in the ventral striatum encode the actual reward size of a gamble, rather than the reward prediction error (the comparison between the expected reward and the actual one).
“An embarrassment of riches?”
Overall, neurons in the ventral striatum encoded not only the expected value of each offer, but also the difference between the expected values of the two offers. Further, they came to represent the selected offer rather than the ignored one; finally, they showed strong outcome-related responses. This behavior is strikingly similar to that of neurons in the ventromedial prefrontal cortex, another cerebral area involved in decision making that the authors previously explored in monkeys as well. Importantly, however, the moment when neurons started to encode the selected choice happened earlier in the ventral striatum than in the ventromedial prefrontal cortex, leading to the intriguing suggestion that value-based decision-making could actually take place in the basal ganglia and not in the cortex. Alternatively, as Jeffrey Stott and David Redish explain in a primer accompanying the research article, these multiple representations of value in the brain could be part of parallel decision-making systems, each one impacting a different aspect of behavior. Whatever the final word on these questions might be, Strait and colleagues’s research has enhanced our understanding of how our brains assess value and make choices based on this assessment.