Using fMRI to Study Valuation and Choice

The ability to make decisions relies on brain mechanisms designed to value our environment and elicit appropriate actions based on those values. These mechanisms allow an agent to predict the value of a potential action both immediately and into the future, and then execute the chosen action. Because the ability to find food or choose a mate directly impacts the survival of a species, it is easy to see how these goal-seeking behaviors would acquire reward value in the brain. But what is the internal value of a piece of art, a label on a bottle, an idea, or a social gesture by another person? Humans use resources (both energy and money) to acquire these types of abstract rewards, and they affect decision-making behavior in a manner similar to primary rewards such as food, water, and sex. In fact, abstract rewards can be powerful motivators: pursuit of these rewards can even cause humans to forego basic needs to acquire them. In this chapter we review experiments in the field of neuroimaging that explore how value is constructed in the human brain across a variety of domains. We then focus on a series of experiments conducted to probe the brain responses underlying preference decisions for art, and how these preferences can be altered by external variables in the environment. These experiments combine neuroscience, psychology, and economics to probe the underlying neurobiology of valuation and choice behavior in humans.


Introduction
The ability to make decisions relies on brain mechanisms designed to value our environment and elicit appropriate actions based on those values. These mechanisms allow an agent to predict the value of a potential action both immediately and into the future, and then execute the chosen action. Because the ability to find food or choose a mate directly impacts the survival of a species, it is easy to see how these goal-seeking behaviors would acquire reward value in the brain. But what is the internal value of a piece of art, a label on a bottle, an idea, or a social gesture by another person? Humans use resources (both energy and money) to acquire these types of abstract rewards, and they affect decision-making behavior in a manner similar to primary rewards such as food, water, and sex. In fact, abstract rewards can be powerful motivators: pursuit of these rewards can even cause humans to forego basic needs to acquire them. In this chapter we review experiments in the field of neuroimaging that explore how value is constructed in the human brain across a variety of domains. We then focus on a series of experiments conducted to probe the brain responses underlying preference decisions for art, and how these preferences can be altered by external variables in the environment. These experiments combine neuroscience, psychology, and economics to probe the underlying neurobiology of valuation and choice behavior in humans.

The study of valuation and choice in the human brain
Like all mobile creatures, a variety of human decisions requires the ability to differentially value choices and their potential outcomes. Many of these decisions relates to the acquisition of food, water, and sex, choices that allow us to survive as a species. In the learning literature, these types of appetitive stimuli are known as primary rewards. Early work on reward processing in alert monkeys demonstrated that midbrain dopaminergic neurons track the delivery of such primary rewards ). Since that time, a large body of ongoing research in this area has provided evidence that dopaminergic target structures are a key component in the brain's reward circuit, and they subsequently respond to an array of primary rewards (Schultz et al, 1997, Breiter et al, 1997, Berns et al, 2001).
However, we make many decisions in our everyday life that have little or no connection to such primary rewards. We make decisions about which brand of paper towels to buy, what to wear, how to spend our afternoon, etc. These types of decisions have components beyond simply meeting our basic needs, and there is often no clear best answer. And yet we routinely make preference judgments and choices for these types of stimuli. Furthermore, we get satisfaction from making these decisions, even in the absence of receiving any primary reward for our efforts. Neuroimaging experiments over the last several years have shed light on the neural mechanisms associated with valuation of abstract rewards. One area that has been shown repeatedly to be activated by diverse rewarding stimuli is the ventromedial prefrontal cortex (VMPFC). This brain region has been shown in several neuroimaging studies to be active for a variety of primary and abstract rewards including sports cars (Erk et al, 2002) (Knutson et al, 2003). Mounting evidence in these studies and others supports the idea that this region is involved in converting the value of these diverse stimuli to a common scale for action selection (Montague and Berns, 2002).

Current research on valuation and choice
The overall goal of this research is to use fMRI to understand how value is constructed for abstract rewards in the human brain. Are value mechanisms for abstract rewards plugging into the same mechanisms dedicated to primary rewards? And if valuation is dynamic, how do context changes modulate responses in these brain areas and subsequently alter the value of an abstract reward? Such a mechanism for comparing value across a set of choices must fit into the existing data on valuation mechanisms for primary rewards, but also extend to the broader context of choices that do not involve primary reward feedback. The series of experiments herein elucidate the neural correlates of valuation for abstract rewards and begin to parameterize ways in which the signal can be modulated by external influences including familiarity and social gestures.

Using art to study valuation
In this chapter we summarize a series of experiments to study subjective decision-making by looking at how humans value visual art. Art represents an interesting category of rewards because they maintain their value even when separated from a primary reinforcer. Despite the fact that that value is a complex variable that depends largely on a person's current internal and external states, humans are incredibly efficient at assigning value to abstract rewards and making appropriate decisions. For our experiments, we assume that expressed behavioral preference for a piece of art represents its internal value. Expressed preference is a common tool used in marketing, psychological, behavioral economics, and imaging experiments to assess the value of rewards. While this is just one example of a subjective decision, it is well-suited to study using fMRI for several reasons. The first reason is simply a practical one: visual art is an easy stimulus to deliver to subjects while inside an MRI scanner. Secondly, people can fairly easily evaluate their preference for art. Even subjects with very little exposure to art can find pieces that they enjoy and pieces that they dislike without having a working knowledge of the art or artist. Third, painting preferences are very unique to the individual. For a given set of paintings, the art that you like/dislike will be different from the art that I like/dislike. This makes the stimulus set more robust for analysis, since we will be unlikely to accidentally bias our analysis toward certain features of the stimuli.
In the first set of experiments, we examined neural responses to abstract rewards using images of paintings as stimuli. Human subjects viewed paintings passively in an fMRI experiment, and were asked to rate their subjective preference and familiarity for every painting after the scanning session ( Figure 1). 80 paintings were chosen to represent a wide array of abstract and representational art from the Western tradition. We first examined the behavioral relationship between preference and familiarity for the paintings (Figure 2A). When we compared the subjective ratings for the two parameters for each painting, we found a striking, positive correlation ( Figure 3.2A, y = 1.04x + 0.086, R 2 = 0.976). Preference varied linearly with familiarity, although of course it is not known if these scales themselves are linear with respect to internal states of preference or familiarity. BOLD responses in several brain regions were found to vary significantly with subjective preference and with familiarity ( Figure 2B). Considerable overlap was found to exist between the two groups, including activation in the ventromedial prefrontal cortex, the bilateral caudate, retrosplenial cortex, and bilateral parahippocampal gyrus, further supporting the relationship between these parameters found behaviorally. The results of the art viewing experiment provide evidence that BOLD responses correlated with painting preference overlap with brain regions known to process a variety of rewarding stimuli. We also demonstrate a strong behavioral and neural correlation between preference and familiarity, suggesting that familiarity is a key factor in the valuation of abstract rewards. These results demonstrate that BOLD responses in several overlapping brain regions encode expressed preference and familiarity for artwork. The region of the vmpfc overlaps with a region previously found to encode preference for colas (McClure et al 2004). This work provides evidence that preferences for abstract stimuli such as art can generate reward responses in the brain similar to primary rewards. The second important finding is that familiarity for these stimuli is encoded in similar brain regions, and may play an important role in determining preferences for art. Notably, we located these neural signatures while subjects passively viewed the artwork in the scanner and were not asked to evaluate the paintings at the time of the viewing. The fact that neural responses during passive viewing correlate with behavioral ratings after the presentation suggest that our brains are doing continuous valuation of stimuli in our environment even in the absence of explicit decisions.

The influence of favors on valuation and decision-making
Subjective decisions may include simple aesthetic choices, but many of our most important decisions are subtle and complex value choices, as in the case of medical decisions and prescribing behavior in the medical and biomedical device industries. Every year, physicians prescribe millions of dollars worth of prescription drugs to treat their patients' illnesses. Often there are comparable drug treatment strategies for a given disease, with slight variations in side effects, cost, etc. How does a physician decide between drug A and drug B? Perhaps in her experience she has found that drug A is more easily tolerated by her patients, or the cost of drug A is lower. But we know that there are other factors at work as well. Each year the pharmaceutical industry directs approximately 90% of its $21 billion marketing budget toward physicians (Brennan et al 2006). This includes sponsored events, lunches, gifts, conferences, and other physician-industry interactions. The effect of such gifts on physician decisionmaking behavior is unmistakable. In several reports on such issues, while physicians routinely deny that gifts could influence their prescribing behavior, and yet receiving samples, industrypaid meals, funding for travel and paid speaking engagements from a pharmaceutical company for a certain drug have all been found to result in more favorable viewpoints and increased prescribing of the particular drug (Wazana, 2000). The fact that physicians are largely unaware of the potential biasing effects of these gestures suggests that external influences are acting in ways that subvert cognitive awareness or control. While biases can be self-serving (Dana and Loewenstein, 2003), they may also be rooted in biological mechanisms that subvert cognitive control. For example, social gestures from a sender to a receiver are known to bias decisions made by the recipient, especially when the decision relates to the sender -a feature of social exchange known as reciprocity We next investigated how external social value can manipulate value judgments for abstract rewards. In the same way that familiarity and preference were tightly linked, perhaps preference judgments depend on other contextual factors. A particularly salient real-world example is the use of monetary favors to influence preferences. We hypothesized that social favors can bias subjective preferences, even for objects unrelated to the favor itself. We have addressed this hypothesis by employing a collection of monetary favor and preference experiments. A monetary favor is sent from an agent to a subject and we tested the influence of this favor on subjects' preference for art, a domain where there is no objectively 'correct' answer and no economic relationship exists between the favor and the preference judgment.
During functional brain imaging sessions, subjects passively viewed images of artwork paired with company logos; one of the companies sponsored the subjects' participation in the experiment (Figure 3). After scanning, the subjects provided preference ratings for all the paintings.
The behavioral results showed that preference for paintings paired with the sponsoring company's logo was higher than those paired with the non-sponsor logo (Harvey et al 2010). The functional imaging results revealed that activity in the ventromedial prefrontal cortex tracked the sponsorship category of the painting (Figure 4, Harvey et al 2010). This imaging study is the first to indicate that a monetary favor transfers value to a proxy and influences judgment of objects in spatial proximity, with corresponding changes in neural activity as measured by fMRI. Furthermore, the experiment was designed with no reciprocal interaction between the company and the receiver, suggesting that the favor is capable of influencing behavior even when the receiver has no explicit stake in the outcome of the interaction.  We hypothesized that social favors can bias subjective preferences, even for objects unrelated to the favor itself. We addressed this hypothesis by employing a collection of monetary favor and revealed preference experiments. Using an art-viewing paradigm possessing no objectively correct answer for preferring one piece of art over another in conjunction with fMRI, we show that sponsorship of the experiment by a company endows the company's logo with the capacity to bias revealed preference for art displayed next to the logo. These sponsorship effects do not possess a special collection of brain responses, but instead modulate responses in neural networks normally activated by a wide range of preference judgments. The results raise the important possibility that monetary favors bias judgments by acting through existing valuation mechanisms, and individuals may therefore have difficulty detecting the gesture's influence over their subjective preferences even for objects seemingly unrelated to the favor.

Mitigation of the effects of favors using domain expertise
In the last series of experiments we examined whether the biasing effects of the social gesture on valuation judgments could be mitigated. In the case of medical professionals, biases from monetary gifts or other favors from pharmaceutical companies is thought to be mitigated through a variety of mechanisms. These include disclosing potential conflicts of interest, not accepting large gifts or favors, and oversight from the institution regarding biases in prescribing behavior. In addition, the fact that medical professionals have expertise in their domain of decision-making is taken as an argument that they should be more objective in their judgment than the average person making a similar decision. To test this directly, we recruited a group of participants with expertise in the domain of art to perform the favor task outlined in the previous section. The hypothesis was that these subjects have training in assessing art that may insulate them against the biasing gesture of the sponsoring company. And if so, there should be neurobiological correlates to the difference in behavior between art experts and non-experts.
Art experts with a minimum of 5 years of experience working in a visual art-related field were asked to undergo an fMRI scan while they viewed images of artwork, paired either with a sponsor company logo or another non-sponsor company logo. After the fMRI session, each subject rated their preference for the art that was displayed in the scanner. The setup was identical to the previously discussed experiments, except that student art was used rather than well known paintings to eliminate the possibility that differences in results between art experts and non-experts was due solely to familiarity with the paintings.
In the behavioral results, there was no difference in preference for the sponsor paintings compared to the non-sponsor paintings, in contrast to the control subjects who showed a significantly higher preference for sponsor paintings. We hypothesized that if the experts did not show an effect of the sponsorship on preference ratings for paintings, BOLD activity in the VMPFC should not be sensitive to the sponsorship in the expert group as originally seen in the control subjects whose behavior reflected the bias toward preferring sponsor paintings. This was indeed the case: while both the controls and art experts had activity in the VMPFC that correlated linearly with painting preference for all paintings, the experts did not show differential activity in this region for sponsor compared to non-sponsor paintings. The next natural question based on the behavioral results was: if art experts do not show an effect of sponsorship on their behavior or activity in the ventromedial prefrontal cortex, which region of the brain might be involved in the mitigation of this effect? To explore this further, we examined activity in the dorsolateral prefrontal cortex (DLPFC), a region known to be involved in executive control (Wagner et al. 2011) and modulation of valuation in other experiments (Hare et al. 2009, Li et al. 2011).
We generated a contrast to look for differences in BOLD response between art experts and controls in the DLPFC for linearly increases preferences for paintings. This region was significantly more active in a linear preference regressor for art experts than controls ( Figure  6). No other brain region showed differential activity for this contrast.
We also found that activity in this region of the DLPFC was functionally connected to the VMPFC, and the coupling of these two regions was stronger during presentation of sponsor paintings than during presentation of non-sponsor paintings. Thus the DLPFC may be a region through which modulation of activity in the VMPFC may occur. Furthermore, we repeated the same functional connectivity analysis in a group of controls that were selected because they did not show a behavioral sponsorship bias. In these participants, we replicated the functional connectivity result between the VMPFC and the DLPFC (Kirk et al 2011). The result indicates that the modulation of the VMPFC by DLPFC may be a more general mechanism by which a person could be insulated by biasing maneuvers, rather than being specific to the individuals with specific domain expertise such as the art experts used in this study.

Future research
In future experiments, we plan to further explore the neurobiology underlying human valuation and behavioral bias, and the ways in which values may be altered with external influences, training, or intervention. There are several lines of questions that lend themselves to the types of experimental manipulations described in this chapter. For example, it may be possible to teach people to train their responses either behaviorally or with real-time feedback during fMRI sessions to become less sensitive to bias. This could be extremely useful for professions where expertise and decision-making should be immune to external influence. One particular profession would be physicians and prescribing behavior: if it is possible to Using fMRI to Study Valuation and Choice http://dx.doi.org/10.5772/47993 assess bias and make changes to their behavior, such intervention may be a much better option than current methods of disclosing conflicts of interest as a method of mitigating bias. Additionally, it is possible that there exist subtypes of people within the population that may be more or less susceptible to bias. Paradigms such as these could be used to parametrically evaluate susceptibility and the effects of intervention. And finally, if a person is biased in one domain are they necessarily biased in another domain as well? Perhaps art experts are not only insulated from bias within the domain of art viewing but across multiple domains that require evaluation without being susceptible to bias. Future research in the areas described will hopefully lead to the discovery of strategies to improve decision-making behavior and knowledge of potential bias across a range of domains.

Conclusion
We have used fMRI in conjunction with behavioral paradigms to investigate the neural correlates of subjective decision-making. Our metric for subjective decisions was behavioral preference for visual art. While this is just one example of a subjective decision, it is well-suited to study using fMRI for several reasons. The first reason is simply a practical one: visual art is an easy stimulus to deliver to subjects while inside an MRI scanner. Secondly, people can fairly easily evaluate their preference for art. Even subjects with very little exposure to art can find pieces that they enjoy and pieces that they dislike without having a working knowledge of the art or artist. Third, painting preferences are very unique to the individual. For a given set of paintings, the art that you like/dislike will be different from the art that I like/dislike. This makes the stimulus set more robust for analysis, since we will be unlikely to accidentally bias our analysis toward certain features of the stimuli. In the first experiment, visual art was passively presented to subjects while they underwent an fMRI scan. We showed that subjects' stated preference for the paintings, made after the scanning session was complete, correlated linearly with the neural responses in several brain regions. The regions with increased BOLD activity included the ventral striatum and the ventromedial prefrontal cortex, regions that are activated by a wide range of rewarding stimuli. In our experiments, however, subjects were not asked to make judgments and were unaware that they would be rating the paintings after the scanner session. This manipulation provides evidence that the brain is likely making continuous valuation judgments even without any explicit query. The results provide further support to the idea that the mechanisms that encode value for primary rewards largely overlap with those for subjective assessments of stimuli with little or no link to such rewards.
In the second set of experiments, we were interested in whether or not external social influences could alter preferences for art. Previous studies have shown that other external factors can alter perception and preferences, with corresponding changes in neural responses. People perceive art as more beautiful when they believe it to be created by an artist compared to a computer (Kirk et al, 2008), the taste of wine is more pleasant (Plassmann et al, 2008)  When a company paid the subject for their participation in the experiment, the visual juxtaposition of the "sponsor" logo made individuals more likely to like those paintings as compared to paintings paired with another, non-sponsor logo. We ran the experiment at three levels of monetary compensation; while there was a trend toward preferring sponsor paintings in all conditions, only the $300 effect was significant. Despite the fact that it took the largest sum of money to influence preferences in this task, it is unlikely that small favors have no biasing effect on behavior. This task, compared to real-world examples of biasing gestures, was relatively mild. The subjects had no option to pay back the company; the gesture was a single-shot interaction rather than a repeated encounter, as is typical of most social gestures. Furthermore, there was absolutely no incentive for subjects to change their preference ratings based on the sponsorship; they were paid exactly the same amount no matter how they answered. The fact that such a mild gesture influences preference judgments completely unrelated to the favor suggests that favors are plugging into reciprocity mechanisms without the individual being aware of the effect. The neural results showed that, rather than a global effect of sponsorship on any brain region, the influence of the sponsor adds linearly to brain responses that more generally encode painting preference.
And finally, we examined the ways in which valuation bias from a favor can be mitigated with domain expertise. We chose art experts for the study because of their experience with the type of stimuli used in the study. We hypothesized that expertise within a domain is one way in which an individual might be insulated by biasing maneuvers such as a monetary favor or other social gesture. While a monetary favor was capable of biasing behavior for a group of control subjects in the previous experiments, the art experts in this study did not show a behavioral preference for sponsor compared to non-sponsor paintings. In the fMRI data, we showed that the VMPFC, a region of the brain known to respond during valuation experiments, was responsive to increasing painting preference for both controls and art experts. However, this region was not differentially sensitive to sponsorship in the art experts, suggesting that the biasing maneuver of the monetary favor was somehow mitigated in this group. We found a candidate region of the dorsolateral prefrontal cortex that may play a role in the mitigation of bias. This region was functionally connected to the VMPFC in the art experts, and also in a group of controls that did not show the sponsorship effect. The results suggest that domain expertise may be one way in which individuals may insulate themselves from bias, but other strategies may be involved for control subjects who show similar effects.
gists who assisted in collecting the fMRI data, the software development team that assisted in coding the task stimuli and development of the Hyperscan technology and NEMO experiment environment. We thank Ron Fisher for early work and collaboration on the art experiment paradigm. We would also like to thank George Denfield for co-author contributions to the published experiments discussed in the chapter (Harvey et al 2010).