%0 Journal Article
%~ PubMed
%A Hogarth, Lee
%A Balleine, Bernard W
%A Corbit, Laura H
%A Killcross, Simon
%T Associative learning mechanisms underpinning the transition from recreational drug use to addiction.
%B Annals of the New York Academy of Sciences
%D 2013
%C United States
%I Wiley-Blackwell Publishing, Inc.
%V 1282
%N 1
%P 12-24
%@ 0077-8923
%X 
%Z FOR Codes: 1109


%0 Journal Article
%~ PubMed
%A Laurent, Vincent
%A Leung, Beatrice
%A Maidment, Nigel
%A Balleine, Bernard W
%T μ- and δ-opioid-related processes in the accumbens core and shell differentially mediate the influence of reward-guided and stimulus-guided decisions on choice.
%B Journal of Neuroscience
%D 2012
%C United States
%I Society for Neuroscience
%V 32
%N 5
%P 1875-1883
%@ 1529-2401
%X Two motivational processes affect choice between actions: (1) changes in the reward value of the goal or outcome of an action and (2) changes in the predicted value of an action based on outcome-related stimuli. Here, we evaluated the role of ??-opioid receptor (MOR) and ??-opioid receptor (DOR) in the nucleus accumbens in the way these motivational processes influence choice using outcome revaluation and pavlovian-instrumental transfer tests. We first examined the effect of genetic deletion of MOR and DOR in specific knock-out mice. We then assessed the effect of infusing the MOR antagonist d-Phe-Cys-Tyr-D-Trp-Arg-Thr-Pen-Thr-NH(2) (CTAP) or the DOR antagonist naltrindole into the core or shell subregions of the nucleus accumbens on these tests in rats. We found that, whereas MOR knock-outs showed normal transfer, they failed to show a selective outcome revaluation effect. Conversely, DOR knock-outs showed normal revaluation but were insensitive to the influence of outcome-related cues on choice. This double dissociation was also found regionally within the nucleus accumbens in rats. Infusion of naltrindole into the accumbens shell abolished transfer but had no effect on outcome revaluation and did not influence either effect when infused into the accumbens core. Conversely, infusion of CTAP into the accumbens core abolished sensitivity to outcome revaluation but had no effect on transfer and did not influence either effect when infused into the accumbens shell. These results suggest that reward-based and stimulus-based values exert distinct motivational influences on choice that can be doubly dissociated both neuroanatomically and neurochemically at the level of the nucleus accumbens.
%Z FOR Codes: 170101 110902


%0 Journal Article
%~ PubMed
%A Lingawi, Nura W
%A Balleine, Bernard W
%T Amygdala central nucleus interacts with dorsolateral striatum to regulate the acquisition of habits.
%B Journal of Neuroscience
%D 2012
%C United States
%I Society for Neuroscience
%V 32
%N 3
%P 1073-1081
%@ 1529-2401
%X The role of the amygdala central nucleus (CeN) in habit learning was assessed in two experiments. First, we examined the effects of bilateral lesions of the anterior CeN on an overtraining-induced lever press habit evaluated using an outcome devaluation protocol. Overtraining generated habitual performance and rendered sham lesioned rats insensitive to outcome devaluation, an effect that was also found in rats given control lesions of the posterior CeN. In contrast, rats with lesions of the anterior CeN did not show normal habit acquisition and their performance remained goal-directed and sensitive to outcome devaluation. Nevertheless, lesions of either the posterior or the anterior CeN abolished the general excitatory influence of a Pavlovian conditioned stimulus on instrumental performance. Second, we assessed the functional interaction between the CeN and dorsolateral striatum (DLS), a region previously implicated in the acquisition of habits, using asymmetrical lesions to disconnect these structures. Rats were given a unilateral lesion of anterior CeN and a unilateral lesion of the DLS, made either ipsilateral (control) or contralateral (disconnection) to the CeN lesion, and given overtraining followed by outcome devaluation. Although the ipsilateral lesioned rats were insensitive to devaluation, the contralateral CeN-DLS lesion impaired habit acquisition, rendering performance sensitive to the devaluation treatment. These results are the first to implicate the CeN and its connection with a circuit involving DLS in habit learning. They imply that, in instrumental conditioning, regions of amygdala parse the instrumental outcome into the reward and reinforcement signals mediating goal-directed and habitual actions, respectively.
%Z FOR Codes: 1109


%0 Journal Article
%~ PubMed
%A Dezfouli, Amir
%A Balleine, Bernard W
%T Habits, action sequences and reinforcement learning.
%B European Journal of Neuroscience
%D 2012
%C United Kingdom
%I Wiley-Blackwell Publishing Ltd.
%V 35
%N 7
%P 1036-1051
%@ 1460-9568
%X It is now widely accepted that instrumental actions can be either goal-directed or habitual; whereas the former are rapidly acquired and regulated by their outcome, the latter are reflexive, elicited by antecedent stimuli rather than their consequences. Model-based reinforcement learning (RL) provides an elegant description of goal-directed action. Through exposure to states, actions and rewards, the agent rapidly constructs a model of the world and can choose an appropriate action based on quite abstract changes in environmental and evaluative demands. This model is powerful but has a problem explaining the development of habitual actions. To account for habits, theorists have argued that another action controller is required, called model-free RL, that does not form a model of the world but rather caches action values within states allowing a state to select an action based on its reward history rather than its consequences. Nevertheless, there are persistent problems with important predictions from the model; most notably the failure of model-free RL correctly to predict the insensitivity of habitual actions to changes in the action-reward contingency. Here, we suggest that introducing model-free RL in instrumental conditioning is unnecessary, and demonstrate that reconceptualizing habits as action sequences allows model-based RL to be applied to both goal-directed and habitual actions in a manner consistent with what real animals do. This approach has significant implications for the way habits are currently investigated and generates new experimental predictions.
%Z FOR Codes: 1109


%0 Book Section
%A O'Doherty, John P.
%A Balleine, Bernard
%T Neural bases of actions and habits
%B Cognitive Search: Evolution, Algorithms, and the Brain
%D 2012
%C United States
%I MIT Press
%V 
%N 
%P 97-110
%@ 9780262018098
%X 
%Z FOR Codes: 1109


%0 Journal Article
%~ PubMed
%A Alvares, Gail A
%A Chen, Nigel T M
%A Balleine, Bernard W
%A Hickie, Ian B
%A Guastella, Adam J
%T Oxytocin selectively moderates negative cognitive appraisals in high trait anxious males.
%B Psychoneuroendocrinology
%D 2012
%C United Kingdom
%I Pergamon
%V 37
%N 12
%P 2022-2031
%@ 1873-3360
%X The mammalian neuropeptide oxytocin has well-characterized effects in facilitating prosocial and affiliative behavior. Additionally, oxytocin decreases physiological and behavioral responses to social stress. In the present study we investigated the effects of oxytocin on cognitive appraisals after a naturalistic social stress task in healthy male students. In a randomized, double-blind, placebo-controlled trial, 48 participants self-administered either an oxytocin or placebo nasal spray and, following a wait period, completed an impromptu speech task. Eye gaze to a pre-recorded video of an audience displayed during the task was simultaneously collected. After the speech, participants completed questionnaires assessing negative cognitive beliefs about speech performance. Whilst there was no overall effect of oxytocin compared to placebo on either eye gaze or questionnaire measures, there were significant positive correlations between trait levels of anxiety and negative self-appraisals following the speech. Exploratory analyses revealed that whilst higher trait anxiety was associated with increasingly poorer perceptions of speech performance in the placebo group, this relationship was not found in participants administered oxytocin. These results provide preliminary evidence to suggest that oxytocin may reduce negative cognitive self-appraisals in high trait anxious males. It adds to a growing body of evidence that oxytocin seems to attenuate negative cognitive responses to stress in anxious individuals.
%Z FOR Codes: 110999 111502 170106


%0 Book Section
%A Winstanley, Catharine A.
%A Redish, A. David
%A Seamans, Jeremy K.
%A Robbins, Trevor W.
%A Balleine, Bernard
%A Brown, Joshua W.
%A Büchel, Christian
%A Cools, Roshan
%A Durstewitz, Daniel
%A O'Doherty, John P.
%A Pennartz, Cyriel M.A.
%T Search, goals and the brain
%B Cognitive Search: Evolution, Algorithms, and the Brain
%D 2012
%C United States
%I MIT Press
%V 
%N 
%P 125-158
%@ 9780262018098
%X 
%Z FOR Codes: 1109


%0 Journal Article
%~ PubMed
%A Bertran-Gonzalez, Jesus
%A Chieng, Billy C
%A Laurent, Vincent
%A Valjent, Emmanuel
%A Balleine, Bernard W
%T Striatal cholinergic interneurons display activity-related phosphorylation of ribosomal protein s6.
%B PLoS One
%D 2012
%C United States
%I Public Library of Science
%V 7
%N 12
%P e53195
%@ 1932-6203
%X 
%Z FOR Codes: 110903


%0 Journal Article
%~ PubMed
%A Wassum, Kate M
%A Tolosa, Vanessa M
%A Tseng, Tina C
%A Balleine, Bernard W
%A Monbouquette, Harold G
%A Maidment, Nigel T
%T Transient extracellular glutamate events in the basolateral amygdala track reward-seeking actions.
%B Journal of Neuroscience
%D 2012
%C United States
%I Society for Neuroscience
%V 32
%N 8
%P 2734-2746
%@ 1529-2401
%X The ability to make rapid, informed decisions about whether or not to engage in a sequence of actions to earn reward is essential for survival. Modeling in rodents has demonstrated a critical role for the basolateral amygdala (BLA) in such reward-seeking actions, but the precise neurochemical underpinnings are not well understood. Taking advantage of recent advancements in biosensor technologies, we made spatially discrete near-real-time extracellular recordings of the major excitatory transmitter, glutamate, in the BLA of rats performing a self-paced lever-pressing sequence task for sucrose reward. This allowed us to detect rapid transient fluctuations in extracellular BLA glutamate time-locked to action performance. These glutamate transients tended to precede lever-pressing actions and were markedly increased in frequency when rats were engaged in such reward-seeking actions. Based on muscimol and tetrodotoxin microinfusions, these glutamate transients appeared to originate from the terminals of neurons with cell bodies in the orbital frontal cortex. Importantly, glutamate transient amplitude and frequency fluctuated with the value of the earned reward and positively predicted lever-pressing rate. Such novel rapid glutamate recordings during instrumental performance identify a role for glutamatergic signaling within the BLA in instrumental reward-seeking actions.
%Z FOR Codes: 1109


%0 Journal Article
%~ PubMed
%A Shiflett, Michael W
%A Balleine, Bernard W
%T Contributions of ERK signaling in the striatum to instrumental learning and performance.
%B Behavioural Brain Research
%D 2011
%C Netherlands
%I Elsevier BV
%V 218
%N 1
%P 240-247
%@ 0166-4328
%X The striatum is critical for learning and decision making; however, the molecular mechanisms that govern striatum function are not fully understood. The extracellular signal regulated kinase (ERK) cascade is an important signaling pathway that underlies synaptic plasticity, cellular excitability, learning and arousal. This review focuses on the role of ERK signaling in striatum function. ERK is activated in the striatum by coordinated dopamine and glutamate receptor signaling, where it underlies corticostriatal synaptic plasticity and influences striatal cell excitability. ERK activation in the dorsal striatum is necessary for action-outcome learning and performance of goal-directed actions. In the ventral striatum, ERK is necessary for the motivating effects of reward-associated stimuli on instrumental performance. Dysregulation of ERK signaling in the striatum by repeated drug exposure contributes to the development of addictive behavior. These results highlight the importance of ERK signaling in the striatum as a critical substrate for learning and as a regulator of ongoing behavior. Furthermore, they suggest that ERK may be a suitable target for therapeutics to treat disorders of learning and decision making that arise from compromised striatum function.
%Z FOR Codes: 110903


%0 Journal Article
%~ PubMed
%A Wassum, Kate M
%A Ostlund, Sean B
%A Balleine, Bernard W
%A Maidment, Nigel T
%T Differential dependence of Pavlovian incentive motivation and instrumental incentive learning processes on dopamine signaling.
%B Learning & Memory
%D 2011
%C United States
%I Cold Spring Harbor Laboratory Press
%V 18
%N 7
%P 475-483
%@ 1549-5485
%X Here we attempted to clarify the role of dopamine signaling in reward seeking. In Experiment 1, we assessed the effects of the dopamine D(1)/D(2) receptor antagonist flupenthixol (0.5 mg/kg i.p.) on Pavlovian incentive motivation and found that flupenthixol blocked the ability of a conditioned stimulus to enhance both goal approach and instrumental performance (Pavlovian-to-instrumental transfer). In Experiment 2 we assessed the effects of flupenthixol on reward palatability during post-training noncontingent re-exposure to the sucrose reward in either a control 3-h or novel 23-h food-deprived state. Flupenthixol, although effective in blocking the Pavlovian goal approach, was without effect on palatability or the increase in reward palatability induced by the upshift in motivational state. This noncontingent re-exposure provided an opportunity for instrumental incentive learning, the process by which rats encode the value of a reward for use in updating reward-seeking actions. Flupenthixol administered prior to the instrumental incentive learning opportunity did not affect the increase in subsequent off-drug reward-seeking actions induced by that experience. These data suggest that although dopamine signaling is necessary for Pavlovian incentive motivation, it is not necessary for changes in reward experience, or for the instrumental incentive learning process that translates this experience into the incentive value used to drive reward-seeking actions, and provide further evidence that Pavlovian and instrumental incentive learning processes are dissociable.
%Z FOR Codes: 1109


%0 Journal Article
%~ PubMed
%A Ostlund, Sean B
%A Wassum, Kate M
%A Murphy, Niall P
%A Balleine, Bernard W
%A Maidment, Nigel T
%T Extracellular dopamine levels in striatal subregions track shifts in motivation and response cost during instrumental conditioning.
%B The Journal of Neuroscience
%D 2011
%C United States
%I Society for Neuroscience
%V 31
%N 1
%P 200-207
%@ 1529-2401
%X Tonic dopamine (DA) signaling is widely regarded as playing a central role in effort-based decision making and in the motivational control of instrumental performance. The current study used microdialysis to monitor changes in extracellular DA levels across subregions of the nucleus accumbens and dorsal striatum of rats as they lever pressed for food reward on a probabilistic schedule of reinforcement, a procedure that ensured they would experience variation in the amount of effort needed to earn rewards across tests. Each rat was given three tests. Rats were hungry for the first and last test, but were sated on food before the middle test, allowing us to assess the effects of a downshift in motivational state on task performance and conditioning-induced DA efflux. During hungry tests, DA levels rose in both the shell and core of the accumbens and, to a lesser degree, in both the medial and lateral divisions of the dorsal striatum. Interestingly, changes in DA efflux across hungry tests in the accumbens core were negatively correlated with changes in the effort required to obtain rewards. We also found that--across regions--the DA response to instrumental conditioning was attenuated when rats were sated before testing. Furthermore, the effect of satiety on DA efflux in the accumbens shell was positively correlated with its effect on task performance. Together, the results indicate that tonic DA contributes to the control of instrumental performance by conveying information about the costs and benefits of responding to different striatal subregions.
%Z FOR Codes: 1109


%0 Journal Article
%~ PubMed
%A Wassum, Kate M
%A Cely, Ingrid C
%A Balleine, Bernard W
%A Maidment, Nigel T
%T Micro-opioid receptor activation in the basolateral amygdala mediates the learning of increases but not decreases in the incentive value of a food reward.
%B The Journal of Neuroscience
%D 2011
%C United States
%I Society for Neuroscience
%V 31
%N 5
%P 1591-1599
%@ 1529-2401
%X The decision to perform, or not perform, actions known to lead to a rewarding outcome is strongly influenced by the current incentive value of the reward. Incentive value is largely determined by the affective experience derived during previous consumption of the reward-the process of incentive learning. We trained rats on a two-lever, seeking-taking chain paradigm for sucrose reward, in which responding on the initial seeking lever of the chain was demonstrably controlled by the incentive value of the reward. We found that infusion of the ?-opioid receptor antagonist, CTOP (d-Phe-Cys-Tyr-d-Trp-Orn-Thr-Pen-Thr-NH(2)), into the basolateral amygdala (BLA) during posttraining, noncontingent consumption of sucrose in a novel elevated-hunger state (a positive incentive learning opportunity) blocked the encoding of incentive value information normally used to increase subsequent sucrose-seeking responses. Similar treatment with ? [N, N-diallyl-Tyr-Aib-Aib-Phe-Leu-OH (ICI 174,864)] or ? [5''-guanidinonaltrindole (GNTI)] antagonists was without effect. Interestingly, none of these drugs affected the ability of the rats to encode a decrease in incentive value resulting from experiencing the sucrose in a novel reduced-hunger state. However, the ? agonist, DAMGO ([d-Ala2, NMe-Phe4, Gly5-ol]-enkephalin), appeared to attenuate this negative incentive learning. These data suggest that upshifts and downshifts in endogenous opioid transmission in the BLA mediate the encoding of positive and negative shifts in incentive value, respectively, through actions at ?-opioid receptors, and provide insight into a mechanism through which opiates may elicit inappropriate desire resulting in their continued intake in the face of diminishing affective experience.
%Z FOR Codes: 1109


%0 Journal Article
%~ PubMed
%A Shiflett, Michael W
%A Balleine, Bernard W
%T Molecular substrates of action control in cortico-striatal circuits.
%B Progress in Neurobiology
%D 2011
%C United Kingdom
%I Pergamon
%V 95
%N 1
%P 1-13
%@ 1873-5118
%X The purpose of this review is to describe the molecular mechanisms in the striatum that mediate reward-based learning and action control during instrumental conditioning. Experiments assessing the neural bases of instrumental conditioning have uncovered functional circuits in the striatum, including dorsal and ventral striatal sub-regions, involved in action-outcome learning, stimulus-response learning, and the motivational control of action by reward-associated cues. Integration of dopamine (DA) and glutamate neurotransmission within these striatal sub-regions is hypothesized to enable learning and action control through its role in shaping synaptic plasticity and cellular excitability. The extracellular signal regulated kinase (ERK) appears to be particularly important for reward-based learning and action control due to its sensitivity to combined DA and glutamate receptor activation and its involvement in a range of cellular functions. ERK activation in striatal neurons is proposed to have a dual role in both the learning and performance factors that contribute to instrumental conditioning through its regulation of plasticity-related transcription factors and its modulation of intrinsic cellular excitability. Furthermore, perturbation of ERK activation by drugs of abuse may give rise to behavioral disorders such as addiction.
%Z FOR Codes: 110903


%0 Journal Article
%~ PubMed
%A Liljeholm, Mimi
%A Tricomi, Elizabeth
%A O'Doherty, John P
%A Balleine, Bernard W
%T Neural correlates of instrumental contingency learning: differential effects of action-reward conjunction and disjunction.
%B The Journal of Neuroscience
%D 2011
%C United States
%I Society for Neuroscience
%V 31
%N 7
%P 2474-2480
%@ 1529-2401
%X Contingency theories of goal-directed action propose that experienced disjunctions between an action and its specific consequences, as well as conjunctions between these events, contribute to encoding the action-outcome association. Although considerable behavioral research in rats and humans has provided evidence for this proposal, relatively little is known about the neural processes that contribute to the two components of the contingency calculation. Specifically, while recent findings suggest that the influence of action-outcome conjunctions on goal-directed learning is mediated by a circuit involving ventromedial prefrontal, medial orbitofrontal cortex, and dorsomedial striatum, the neural processes that mediate the influence of experienced disjunctions between these events are unknown. Here we show differential responses to probabilities of conjunctive and disjunctive reward deliveries in the ventromedial prefrontal cortex, the dorsomedial striatum, and the inferior frontal gyrus. Importantly, activity in the inferior parietal lobule and the left middle frontal gyrus varied with a formal integration of the two reward probabilities, ?P, as did response rates and explicit judgments of the causal efficacy of the action.
%Z FOR Codes: 1109


%0 Book Section
%A Balleine, Bernard
%T Sensation, incentive learning, and the motivational control of goal-directed action
%B Neurobiology of Sensation and Reward
%D 2011
%C United States
%I Taylor and Francis
%V 
%N 
%P 287-310
%@ 9781420067262
%E Gottfried, Jay A
%X 
%Z FOR Codes: 110906


%0 Journal Article
%~ PubMed
%A Corbit, Laura H
%A Balleine, Bernard W
%T The general and outcome-specific forms of pavlovian-instrumental transfer are differentially mediated by the nucleus accumbens core and shell.
%B The Journal of  Neuroscience
%D 2011
%C United States
%I Society for Neuroscience
%V 31
%N 33
%P 11786-11794
%@ 1529-2401
%X Tests of Pavlovian-instrumental transfer (PIT) demonstrate that reward-predictive stimuli can exert a powerful motivational influence on the performance of instrumental actions. Recent evidence suggests that predictive stimuli produce this effect through either the general arousal (general PIT) or the specific predictions (outcome-specific PIT) produced by their association with reward. In two experiments, we examined the effects of pretraining lesions (Experiment 1) or muscimol-induced inactivation (Experiment 2) of either the core or shell regions of the nucleus accumbens (NAC) on these forms of PIT. Rats received Pavlovian training in which three auditory stimuli each predicted the delivery of a distinct food outcome. Separately, the rats were trained to perform two instrumental actions, each of which earned one of the outcomes used in Pavlovian conditioning. Finally, the effects of the three stimuli on performance of the two actions were assessed in extinction. Here we report evidence of a double dissociation between general and outcome-specific PIT at the level of the accumbens. Shell lesions eliminated outcome-specific PIT but spared general PIT, whereas lesions of the core abolished general PIT but spared outcome-specific PIT. Importantly, the infusion of muscimol into core or shell made immediately before the PIT tests produced a similar pattern of results. These results suggest that whereas the NAC core mediates the general excitatory effects of reward-related cues, the NAC shell mediates the effect of outcome-specific reward predictions on instrumental performance, and thereby serve to clarify reported discrepancies regarding the role of the NAC core and shell in PIT.
%Z FOR Codes: 1109


%0 Journal Article
%~ PubMed
%A Balleine, Bernard W
%A Leung, Beatrice K
%A Ostlund, Sean B
%T The orbitofrontal cortex, predicted value, and choice.
%B Annals of the New York Academy of Sciences
%D 2011
%C United States
%I Wiley-Blackwell Publishing, Inc.
%V 1239
%N 1
%P 43-50
%@ 0077-8923
%X Considerable evidence suggests that choice between goal-directed actions depends on two incentive processes encoding the reward value of the goal or outcome and the predicted value of an action based on outcome-related stimuli. Although incentive theories generally assume that these processes are mediated by a common associative mechanism, a number of recent findings suggest that they are dissociable; the reward value of an action is derived from consummatory experience with the outcome itself, whereas the predicted value of an action is based on the presence of outcome-associated stimuli from which estimates of the likelihood of an outcome are derived. Importantly, the orbitofrontal cortex (OFC) in rodents appears to mediate the effect of outcome-related stimuli on choice; OFC lesions disrupt the influence of Pavlovian stimuli on choice in tests of outcome-specific Pavlovian-instrumental transfer. However, the influence of outcome-related stimuli on choice involves a larger circuit including the OFC, the ventral striatum, and the amygdala. How these structures interact, however, is not yet fully understood and is an important question for future research.
%Z FOR Codes: 1109


%0 Journal Article
%~ PubMed
%A Shiflett, Michael W
%A Brown, Robert A
%A Balleine, Bernard W
%T Acquisition and performance of goal-directed instrumental actions depends on ERK signaling in distinct regions of dorsal striatum in rats.
%B The Journal of neuroscience : the official journal of the Society for Neuroscience
%D 2010
%C United States
%I Society for Neuroscience
%V 30
%N 8
%P 2951-2959
%@ 1529-2401
%X The performance of goal-directed actions relies on an animal''s previous knowledge of the outcomes or consequences that result from its actions. Additionally, a sensorimotor learning process linking environmental stimuli with actions influences instrumental performance by selecting actions for additional evaluation. These distinct decision-making processes in rodents depend on separate subregions of the dorsal striatum. Whereas the posterior dorsomedial striatum (pDMS) is required for the encoding of actions with their outcomes or consequences, the dorsolateral striatum (DLS) mediates action selection based on sensorimotor learning. However, the molecular mechanisms within these brain regions that support learning and performance of goal-directed behavior are not known. Here we show that activation of extracellular signal-regulated kinase (ERK) in the dorsal striatum has a critical role in learning and performance of instrumental goal-directed behavior in rodents. We observed an increase in p42 ERK (ERK2) activation in both the pDMS and DLS during both the acquisition and performance of recently acquired instrumental goal-directed actions. Furthermore, disruption of ERK activation in the pDMS prevented both the acquisition of action-outcome associations, as well as the performance of goal-directed actions guided by previously acquired associations, whereas disruption of ERK activation in the DLS disrupted instrumental performance but left instrumental action-outcome learning intact. These results provide evidence of a critical, region-specific role for ERK signaling in the dorsal striatum during the acquisition of instrumental learning and suggest that processes sensitive to ERK signaling within these striatal subregions interact to control instrumental performance after initial acquisition.
%Z FOR Codes: 110903


%0 Journal Article
%~ PubMed
%A Ostlund, Sean B
%A Maidment, Nigel T
%A Balleine, Bernard W
%T Alcohol-Paired Contextual Cues Produce an Immediate and Selective Loss of Goal-directed Action in Rats.
%B Frontiers in integrative neuroscience
%D 2010
%C Switzerland
%I Frontiers Research Foundation
%V 4
%N 
%P pii: 19
%@ 1662-5145
%X We assessed whether the presence of contextual cues paired with alcohol would disrupt rats'' capacity to express appropriate goal-directed action control. Rats were first given differential context conditioning such that one set of contextual cues was paired with the injection of ethanol and a second, distinctive set of cues was paired with the injection of saline. All rats were then trained in a third, neutral context to press one lever for grain pellets and another lever for sucrose pellets. They were then given two extinction tests to evaluate their ability to choose between the two actions in response to the devaluation of one of the two food outcomes with one test conducted in the alcohol-paired context and the other conducted in the control (saline-paired) context. In the control context, rats exhibited goal-directed action control; i.e., they were able selectively to withhold the action that previously earned the now devalued outcome. However, these same rats were impaired when tested in the alcohol-paired context, performing both actions at the same rate regardless of the current value of their respective outcomes. Subsequent testing revealed that the rats were capable of overcoming this impairment if they were giving response-contingent feedback about the current value of the food outcomes. These results provide a clear demonstration of the disruptive influence that alcohol-paired cues can exert on decision-making in general and goal-directed action selection and choice in particular.
%Z FOR Codes: 110906


%0 Journal Article
%~ PubMed
%A Shiflett, Michael W
%A Balleine, Bernard W
%T At the limbic-motor interface: disconnection of basolateral amygdala from nucleus accumbens core and shell reveals dissociable components of incentive motivation.
%B The European journal of neuroscience
%D 2010
%C United Kingdom, Germ
%I Wiley-Blackwell Publishing Ltd.
%V 32
%N 10
%P 1735-43
%@ 1460-9568
%X Although it has long been hypothesized that the nucleus accumbens (NAc) acts as an interface between limbic and motor regions, direct evidence for this modulatory role on behavior is lacking. Using a disconnection procedure in rats, we found that basolateral amygdala (BLA) input to the core and medial shell of the NAc separately mediate two distinct incentive processes controlling the performance of goal-directed instrumental actions, respectively: (i) the sensitivity of instrumental responding to changes in the experienced value of the goal or outcome, produced by specific satiety-induced outcome devaluation; and (ii) the effect of reward-related cues on action selection, observed in outcome-specific Pavlovian-instrumental transfer. These results reveal, therefore, that dissociable neural circuits involving BLA inputs to the NAc core and medial shell mediate distinct components of the incentive motivational processes controlling choice and decision-making in instrumental conditioning.
%Z FOR Codes: 110903


%0 Journal Article
%~ PubMed
%A Liljeholm, Mimi
%A Balleine, Bernard W
%T Extracting functional equivalence from reversing contingencies.
%B Journal of experimental psychology. Animal behavior processes
%D 2010
%C United States
%I American Psychological Association
%V 36
%N 2
%P 165-171
%@ 1939-2184
%X In 2 experiments assessing acquired equivalence, human participants were initially presented with 4 cues, 2 of which were paired with 1 outcome and 2 of which were paired with a 2nd outcome. These contingencies were then reversed across several training blocks such that, although each cue was paired equally often with each of the two outcomes across blocks, cues A and B always signaled the same outcome within blocks (as did cues C and D). In both experiments, performance on a subsequent transfer discrimination was enhanced when participants were required to generalize between stimuli that had been paired with the same outcome within each block of training. Additional tests did not yield evidence of a bias toward a specific set of cue-outcome contingencies in either experiment. Moreover, interviews conducted at the end of Experiment 2 revealed that performance on the transfer discrimination was enhanced only in participants who discovered the equivalence relationships during initial training. The results challenge simple associative, and attentional, accounts of acquired equivalence and favor the view that this effect is mediated by comparisons of the similarity of adjacent cue-outcome structures.
%Z FOR Codes: 110903


%0 Journal Article
%~ PubMed
%A Balleine, Bernard W
%A O'Doherty, John P
%T Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action.
%B Neuropsychopharmacology
%D 2010
%C United Kingdom, Unit
%I Nature Publishing Group
%V 35
%N 1
%P 48-69
%@ 1470-634X
%X Recent behavioral studies in both humans and rodents have found evidence that performance in decision-making tasks depends on two different learning processes; one encoding the relationship between actions and their consequences and a second involving the formation of stimulus-response associations. These learning processes are thought to govern goal-directed and habitual actions, respectively, and have been found to depend on homologous corticostriatal networks in these species. Thus, recent research using comparable behavioral tasks in both humans and rats has implicated homologous regions of cortex (medial prefrontal cortex/medial orbital cortex in humans and prelimbic cortex in rats) and of dorsal striatum (anterior caudate in humans and dorsomedial striatum in rats) in goal-directed action and in the control of habitual actions (posterior lateral putamen in humans and dorsolateral striatum in rats). These learning processes have been argued to be antagonistic or competing because their control over performance appears to be all or none. Nevertheless, evidence has started to accumulate suggesting that they may at times compete and at others cooperate in the selection and subsequent evaluation of actions necessary for normal choice performance. It appears likely that cooperation or competition between these sources of action control depends not only on local interactions in dorsal striatum but also on the cortico-basal ganglia network within which the striatum is embedded and that mediates the integration of learning with basic motivational and emotional processes. The neural basis of the integration of learning and motivation in choice and decision-making is still controversial and we review some recent hypotheses relating to this issue.Neuropsychopharmacology advance online publication, 23 September 2009; doi:10.1038/npp.2009.131.
%Z FOR Codes: 1109 1116


%0 Journal Article
%~ PubMed
%A Tricomi, Elizabeth
%A Balleine, Bernard W
%A O'Doherty, John P
%T A specific role for posterior dorsolateral striatum in human habit learning.
%B The European journal of neuroscience
%D 2009
%C United Kingdom
%I Blackwell Publishing Ltd
%V 29
%N 11
%P 2225-2232
%@ 0953-816X
%X Habits are characterized by an insensitivity to their consequences and, as such, can be distinguished from goal-directed actions. The neural basis of the development of demonstrably outcome-insensitive habitual actions in humans has not been previously characterized. In this experiment, we show that extensive training on a free-operant task reduces the sensitivity of participants'' behavior to a reduction in outcome value. Analysis of functional magnetic resonance imaging data acquired during training revealed a significant increase in task-related cue sensitivity in a right posterior putamen-globus pallidus region as training progressed. These results provide evidence for a shift from goal-directed to habit-based control of instrumental actions in humans, and suggest that cue-driven activation in a specific region of dorsolateral posterior putamen may contribute to the habitual control of behavior in humans.
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Wassum, K M
%A Cely, I C
%A Maidment, N T
%A Balleine, B W
%T Disruption of endogenous opioid activity during instrumental learning enhances habit acquisition.
%B Neuroscience
%D 2009
%C United Kingdom
%I Pergamon
%V 163
%N 3
%P 770-80
%@ 0306-4522
%X Considerable evidence suggests that in instrumental conditioning rats learn the relationship between actions and their consequences, or outcomes. Such goal-directed actions are sensitive to changes in outcome value. The present study assessed the role of the endogenous opioid system in goal-directed reward learning. In two experiments, rats were trained to lever press for food pellets either under vehicle or naloxone-induced opioid receptor blockade. Specific satiety procedures were used for outcome devaluation, and the effect of this devaluation on instrumental responding was then tested in extinction. In Experiment 1 outcome devaluation resulted in a reduction in lever pressing in rats that were trained after vehicle injections, indicating that actions in these rats were goal-directed. In contrast, actions in rats trained under naloxone were insensitive to outcome devaluation when tested off drug, suggesting that lever pressing had become habitual in these rats. Interestingly, in Experiment 2 naloxone-induced habitual behavior was shown to be specific to the context in which the training occurred under naloxone; rats showed normal sensitivity to outcome devaluation when tested in an alternate vehicle-trained context. Additionally, in Experiment 2 we found that the acute administration of naloxone on test had no effect in itself, indicating that opioid receptor-related processes contribute to the acquisition of goal-directed actions and not to their general performance. These data suggest that an intact endogenous opioid system is necessary for normal goal-directed learning and more importantly, reveal that a compromised endogenous opioid system during learning enhances the habitual control of actions.
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Wassum, K M
%A Ostlund, S B
%A Maidment, N T
%A Balleine, B W
%T Distinct opioid circuits determine the palatability and the desirability of rewarding events.
%B Proceedings of the National Academy of Sciences of the United States of America
%D 2009
%C United States
%I Natl Acad Sciences
%V 106
%N 30
%P 12512-12517
%@ 1091-6490
%X It generally is assumed that a common neural substrate mediates both the palatability and the reward value of nutritive events. However, recent evidence suggests this assumption may not be true. Whereas opioid circuitry in both the nucleus accumbens and ventral pallidum has been reported to mediate taste-reactivity responses to palatable events, the assignment of reward or inventive value to goal-directed actions has been found to involve the basolateral amygdala. Here we found that, in rats, the neural processes mediating palatability and incentive value are indeed dissociable. Naloxone infused into either the ventral pallidum or nucleus accumbens shell blocked the increase in sucrose palatability induced by an increase in food deprivation without affecting the performance of sucrose-related actions. Conversely, naloxone infused into the basolateral amygdala blocked food deprivation-induced changes in sucrose-related actions without affecting sucrose palatability. This double dissociation of opioid-mediated changes in palatability and incentive value suggests that the role of endogenous opioids in reward processing does not depend on a single neural circuit. Rather, changes in palatability and in the incentive value assigned to rewarding events seem to be mediated by distinct neural processes.
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Ostlund, Sean B
%A Winterbauer, Neil E
%A Balleine, Bernard W
%T Evidence of action sequence chunking in goal-directed instrumental conditioning and its dependence on the dorsomedial prefrontal cortex.
%B The Journal of neuroscience
%D 2009
%C United States
%I Society for Neuroscience
%V 29
%N 25
%P 8280-8287
%@ 1529-2401
%X The current study investigated the contribution of the dorsomedial prefrontal cortex (dmPFC) to instrumental action selection. We found that cell body lesions of the dmPFC, centered on the medial agranular area, spared rats'' ability to choose between actions based on either the value or the discriminative stimulus properties of an outcome. We next examined the effects of these lesions on action sequence learning using a concurrent bidirectional heterogeneous chain task in which the identity of the reward delivered was determined by the order in which the two lever press actions were performed. Although both lesioned rats and sham controls learned to perform the task, we found that they relied on different behavioral strategies to do so. In subsequent tests, rats in the sham group were able to withhold their performance of a sequence when either its associated outcome was devalued or the contingency between that sequence and its outcome was degraded by delivering the outcome noncontingently. Interestingly, lesioned rats failed to reorganize their performance at the action sequence level and, rather, were found to withhold their performance of the terminal response in the sequence that had earned the devalued outcome relative to the more distal response, suggesting that they represented the elements of the sequence as distinct behavioral units. These findings demonstrate that rats can use sequence-level representations, or action chunks, to organize their behavior in a goal-directed manner and indicate that the dmPFC plays a critical role in this process.
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Tanaka, Saori C
%A Balleine, Bernard W
%A O'Doherty, John P
%T Calculating consequences: brain systems that encode the causal effects of actions.
%B The Journal of neuroscience
%D 2008
%C United States
%I Society for Neuroscience
%V 28
%N 26
%P 6750-6755
%@ 1529-2401
%X The capacity to accurately evaluate the causal effectiveness of our actions is key to successfully adapting to changing environments. Here we scanned subjects using functional magnetic resonance imaging while they pressed a button to earn money as the response-reward relationship changed over time. Subjects'' judgments about the causal efficacy of their actions reflected the objective contingency between the rate of button pressing and the amount of money they earned. Neural responses in medial orbitofrontal cortex and dorsomedial striatum were modulated as a function of contingency, by increasing in activity during sessions when actions were highly causal compared with when they were not. Moreover, medial prefrontal cortex tracked local changes in action-outcome correlations, implicating this region in the on-line computation of contingency. These results reveal the involvement of distinct brain regions in the computational processes that establish the causal efficacy of actions, providing insight into the neural mechanisms underlying the adaptive control of behavior.
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Ostlund, Sean B
%A Balleine, Bernard W
%T Differential involvement of the basolateral amygdala and mediodorsal thalamus in instrumental action selection.
%B The Journal of neuroscience
%D 2008
%C United States
%I Society for Neuroscience
%V 28
%N 17
%P 4398-4405
%@ 1529-2401
%X Although it has been shown that the basolateral amygdala (BLA) and the mediodorsal thalamus (MD) are critical for goal-directed instrumental performance, much remains unknown about the respective contributions of these structures to action selection. The current study assessed the effects of post-training BLA and MD lesions on several tests of instrumental action selection. We found that MD damage disrupted the influence of pavlovian cues over action selection but left intact rats'' ability to select actions based on either the expected value or the discriminative stimulus properties of the outcome. In contrast, BLA lesions impaired performance on all three tests of action selection. Because both lesion types disrupted the influence of cues that signal reward over instrumental performance, we then investigated the involvement of these structures in pavlovian contingency learning using a task in which the predictive status of one of two cues is degraded by delivering its outcome noncontingently during the intertrial interval. As expected, the sham group selectively suppressed their conditioned approach performance to the cue that no longer signaled its outcome but continued to respond to the control stimulus. In contrast, both lesioned groups were impaired on this task. Interestingly, whereas the MD group displayed a nonspecific reduction in responding to both cues, the BLA group continued to show high levels of responding to both cues as if their performance was completely insensitive to this contingency manipulation. These findings demonstrate that the BLA and MD make important yet distinct contributions to instrumental action selection.
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Espinet, Alfredo
%A Artigas, Antonio A
%A Balleine, Bernard W
%T Inhibitory sensory preconditioning detected with a sodium depletion procedure.
%B Quarterly journal of experimental psychology
%D 2008
%C United States
%I Psychology Press
%V 61
%N 2
%P 240-247
%@ 1747-0218
%X In each of two experiments, two groups of rats were exposed to three flavoured solutions: A (citric acid), B (salt), and AX (a compound of citric acid and saccharin). Both experiments used a between-subjects design in which a paired group received presentations of A followed by B, alternating with presentations of AX (i.e., A --> B/AX), and an unpaired group received alternating presentations of A, B, and AX (i.e., A/B/AX). This arrangement was expected to establish X as an inhibitor of B in group paired but not in group unpaired. In Experiment 1, after preexposure all subjects received a single presentation of an XB compound, then experienced sodium depletion, and were tested for their consumption of X, which was greater in group unpaired than in group paired. In Experiment 2, after preexposure, all subjects received four presentations of a new flavour, C, in compound with B and subsequently, under sodium depletion, were tested for consumption of XC. Intake of the XC compound was less in group paired than in group unpaired. These results suggest that, in group paired, X acquired an inhibitory relationship with B both retarding the acquisition of an excitatory association with B (retardation test, Experiment 1) and reducing the response to a new stimulus, C, strongly associated to B (summation test, Experiment 2). These results provide direct evidence of inhibition between two neutral stimuli and, therefore, of inhibitory sensory preconditioning.
%Z FOR Codes: 110999


%0 Book Section
%A Balleine, Bernard
%A Daw, N D
%A O'Doherty, J
%T Multiple forms of value learning and the function of dopamine
%B Neuroeconomics
%D 2008
%C United Kingdom
%I Academic Press
%V 
%N 
%P 367-387
%@ 9780123741769
%E Glimcher, Paul W
%E Camerer, Colin
%E Poldrack, Russell Alan
%E Fehr, Ernst
%X 
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Ostlund, Sean B
%A Balleine, Bernard W
%T On habits and addiction: An associative analysis of compulsive drug seeking.
%B Drug discovery today. Disease models
%D 2008
%C United Kingdom
%I Elsevier Ltd. * Trends Journals
%V 5
%N 4
%P 235-245
%@ 1740-6757
%X The processes that underlie the pathological pursuit of drugs in addiction and that support the transition from casual drug taking to their compulsive pursuit have recently been proposed to reflect the interaction of two action control processes that mediate the goal-directed and habitual control of actions for natural rewards. Here we describe the evidence for these learning processes, their associate structure and the motivational mechanisms through which their operation is translated into performance. Finally, we describe the potential changes in the interaction between habitual and goal-directed processes induced by drug addiction that subserve compulsive drug pursuit; i.e. the increase in habit learning and reduction in the regulation of habits induced by changes in the circuitry that mediates goal-directed action.
%Z FOR Codes: 110902


%0 Journal Article
%~ PubMed
%A Yin, Henry H
%A Ostlund, Sean B
%A Balleine, Bernard W
%T Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks.
%B The European journal of neuroscience
%D 2008
%C United Kingdom
%I Blackwell Publishing Ltd
%V 28
%N 8
%P 1437-1448
%@ 0953-816X
%X Here we challenge the view that reward-guided learning is solely controlled by the mesoaccumbens pathway arising from dopaminergic neurons in the ventral tegmental area and projecting to the nucleus accumbens. This widely accepted view assumes that reward is a monolithic concept, but recent work has suggested otherwise. It now appears that, in reward-guided learning, the functions of ventral and dorsal striata, and the cortico-basal ganglia circuitry associated with them, can be dissociated. Whereas the nucleus accumbens is necessary for the acquisition and expression of certain appetitive Pavlovian responses and contributes to the motivational control of instrumental performance, the dorsal striatum is necessary for the acquisition and expression of instrumental actions. Such findings suggest the existence of multiple independent yet interacting functional systems that are implemented in iterating and hierarchically organized cortico-basal ganglia networks engaged in appetitive behaviors ranging from Pavlovian approach responses to goal-directed instrumental actions controlled by action-outcome contingencies.
%Z FOR Codes: 110999


%0 Book Section
%A Balleine, Bernard
%T Taste, disgust and value: taste aversion learning and outcome encoding in instrumental conditioning
%B Conditioned Taste Aversion: Neural and Behavioral Processes
%D 2008
%C United States
%I Oxford University Press
%V 
%N 
%P 262-280
%@ 9780195326581
%E Reilly, Steve
%E Schachtman, Todd R
%X 
%Z FOR Codes: 110906


%0 Journal Article
%A Ostlund, S B
%A Balleine, Bernard
%T The disunity of Pavlovian and instrumental values
%B Behavioral and Brain Sciences
%D 2008
%C United Kingdom
%I Cambridge University Press
%V 31
%N 
%P 456-457
%@ 0140-525X
%X 
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Bray, Signe
%A Rangel, Antonio
%A Shimojo, Shinsuke
%A Balleine, Bernard
%A O'Doherty, John P
%T The neural mechanisms underlying the influence of pavlovian cues on human decision making.
%B The Journal of neuroscience
%D 2008
%C United States
%I Society for Neuroscience
%V 28
%N 22
%P 5861-5866
%@ 1529-2401
%X In outcome-specific transfer, pavlovian cues that are predictive of specific outcomes bias action choice toward actions associated with those outcomes. This transfer occurs despite no explicit training of the instrumental actions in the presence of pavlovian cues. The neural substrates of this effect in humans are unknown. To address this, we scanned 23 human subjects with functional magnetic resonance imaging while they made choices between different liquid food rewards in the presence of pavlovian cues previously associated with one of these outcomes. We found behavioral evidence of outcome-specific transfer effects in our subjects, as well as differential blood oxygenation level-dependent activity in a region of ventrolateral putamen when subjects chose, respectively, actions consistent and inconsistent with the pavlovian-predicted outcome. Our results suggest that choosing an action incompatible with a pavlovian-predicted outcome might require the inhibition of feasible but nonselected action-outcome associations. The results of this study are relevant for understanding how marketing actions can affect consumer choice behavior as well as for how environmental cues can influence drug-seeking behavior in addiction.
%Z FOR Codes: 110999


%0 Book Section
%A Ostlund, S B
%A Winterbauer, N E
%A Balleine, Bernard
%T Theory of Reward Systems
%B Learning Theory and Behavior. Learning and Memory: A Comprehensive Reference, Vol 1.
%D 2008
%C United Kingdom
%I Elsevier
%V 
%N 
%P 701-720
%@ 9780123705044
%E Menzel, R
%X 
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Nader, Karim
%A Balleine, Bernard
%T Ambiguity and anxiety: when a glass half full is empty.
%B Nature Neuroscience
%D 2007
%C United States
%I Nature Publishing Group
%V 10
%N 7
%P 807-808
%@ 1097-6256
%X 
%Z FOR Codes: 1109


%0 Journal Article
%~ PubMed
%A Balleine, Bernard W
%A Doya, Kenji
%A O'Doherty, John
%A Sakagami, Masamichi
%T Current trends in decision making.
%B Annals of the New York Academy of Sciences
%D 2007
%C United States
%I New York Acad Sciences
%V 1104
%N 
%P xi-xv
%@ 0077-8923
%X 
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Corbit, Laura H
%A Janak, Patricia H
%A Balleine, Bernard W
%T General and outcome-specific forms of Pavlovian-instrumental transfer: the effect of shifts in motivational state and inactivation of the ventral tegmental area.
%B The European journal of neuroscience
%D 2007
%C United Kingdom
%I Blackwell Publishing Ltd
%V 26
%N 11
%P 3141-3149
%@ 0953-816X
%X This study compared the contribution of the general activating and specific cueing properties of Pavlovian stimuli to Pavlovian-instrumental transfer (PIT) and the role of the ventral tegmental area (VTA) in mediating these effects. In Experiment 1, hungry rats initially received Pavlovian training, in which three distinct auditory stimuli predicted the delivery of three different food outcomes. Next, the rats were trained to perform two instrumental actions, each earning a unique outcome selected from the three used in Pavlovian conditioning. Finally, the effects of the three stimuli on performance of the two actions were assessed in extinction. Presentation of a stimulus that had been paired with the same outcome as an action increased its performance relative to the other action, demonstrating that PIT effects can be outcome selective. In contrast, presentation of the stimulus that predicted the outcome that was not earned during instrumental training facilitated the performance of both actions indiscriminately. This effect, but not the outcome-selective effect, was abolished by a shift from a hungry to a relatively sated state. Experiment 2 examined the effects of inactivation of the VTA on these two forms of PIT. VTA inactivation was found to attenuate PIT but, unlike satiety, did not appear to differentially affect the general or the outcome-selective forms of PIT. The VTA appears therefore to play an important but general role in the initiation of instrumental actions, enabling cues to influence performance whether they enhance responding by changes in arousal or by retrieving particular actions based on their consequences.
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Lobo, Mary Kay
%A Cui, Yijun
%A Ostlund, Sean B
%A Balleine, Bernard W
%A Yang, X William
%T Genetic control of instrumental conditioning by striatopallidal neuron-specific S1P receptor Gpr6.
%B Nature neuroscience
%D 2007
%C United States
%I Nature Publishing Group
%V 10
%N 11
%P 1395-1397
%@ 1097-6256
%X Instrumental conditioning allows animals to learn about the consequences of their own actions, but the underpinning molecular mechanisms remain elusive. Here we show that the sphingosine-1-phosphate (S1P) receptor Gpr6 is selectively expressed in the striatopallidal neurons in the striatum. Gpr6-deficient mice showed reduced striatal cyclic AMP production in vitro and selective alterations in instrumental conditioning in vivo. Thus, Gpr6 is the first striatopallidal neuron-specific genetic regulator of instrumental conditioning in a mammal.
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Ostlund, Sean B
%A Balleine, Bernard W
%T Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental conditioning.
%B The Journal of neuroscience
%D 2007
%C United States
%I Society for Neuroscience
%V 27
%N 18
%P 4819-4825
%@ 1529-2401
%X Previous studies have implicated the orbitofrontal cortex (OFC) in outcome encoding. However, it remains unknown whether the OFC is selectively involved in pavlovian stimulus-outcome learning or whether it also contributes to instrumental action-outcome learning. In experiment 1, we investigated this issue by assessing the effects of bilateral lesions of the OFC on the sensitivity of instrumental lever press performance to a reduction in the incentive value of the training outcome (a test of action-outcome encoding) and to outcome-specific pavlovian-instrumental transfer (a test of stimulus-outcome encoding). We found that post-training lesions of the OFC did not affect instrumental outcome devaluation, but abolished the transfer effect. Interestingly, lesions made before training had no effect on either task. In experiment 2, we explored the involvement of the OFC in updating stimulus-outcome associations after the underlying contingency, or predictive relationship, between these two events has been degraded. Shams displayed clear contingency learning, withholding conditioned responding to a stimulus that no longer reliably predicted its outcome while continuing to respond to a control stimulus that remained a good predictor of a different outcome. In contrast, OFC-lesioned rats stopped responding to both stimuli, regardless of their predictive status. Together, these findings suggest that the OFC supports outcome encoding in pavlovian, but not instrumental conditioning.
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Ostlund, Sean B
%A Balline, Bernard W
%T Selective reinstatement of instrumental performance depends on the discriminative stimulus properties of the mediating outcome.
%B Learning & behavior
%D 2007
%C United States
%I Psychonomic Society, Inc
%V 35
%N 1
%P 43-52
%@ 1543-4494
%X We conducted three experiments to investigate the associative structure underlying the reinstatement of instrumental performance after extinction. In each experiment, rats were initially rewarded on two responses with different outcomes. At test, both responses were extinguished in order to assess the impact of a single noncontingent outcome delivery on response selection. Experiment 1 found evidence of outcome-selective reinstatement (i.e., more responses were performed on the lever that was trained with the reinstating outcome than on the other lever). Experiment 2 demonstrated that the outcome''s capacity to reinstate performance was not affected by a reduction in its motivational value. Experiment 3 found evidence that the reinstating outcome selectively retrieved the response it signaled rather than the response it followed during training. Together, these findings are consistent with the view that instrumental reinstatement depends on the discriminative stimulus properties of the reinstating outcome.
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Balleine, Bernard W
%A Ostlund, Sean B
%T Still at the choice-point: action selection and initiation in instrumental conditioning.
%B Annals of the New York Academy of Sciences
%D 2007
%C United States
%I New York Acad Sciences
%V 1104
%N 
%P 147-171
%@ 0077-8923
%X Contrary to classic stimulus-response (S-R) theory, recent evidence suggests that, in instrumental conditioning, rats encode the relationship between their actions and the specific consequences that these actions produce. It has remained unclear, however, how encoding this relationship acts to control instrumental performance. Although S-R theories were able to give a clear account of how learning translates into performance, the argument that instrumental learning constitutes the acquisition of information of the form "response R leads to outcome O" does not directly imply a particular performance rule or policy; this information can be used both to perform R and to avoid performing R. Recognition of this problem has forced the development of accounts that allow the O and stimuli that predict the O (i.e., S-O) to play a role in the initiation of specific Rs. In recent experiments, we have used a variety of behavioral procedures in an attempt to isolate the processes that contribute to instrumental performance, including outcome devaluation, reinstatement, and Pavlovian-instrumental transfer. Our results, particularly from experiments assessing outcome-selective reinstatement, suggest that both "feed-forward" (O-R) and "feed-back" (R-O) associations are critical and that although the former appear to be important to response selection, the latter-together with processes that determine outcome value-mediate response initiation. We discuss a conceptual model that integrates these processes and its neural implementation.
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Ostlund, Sean B
%A Balleine, Bernard W
%T The contribution of orbitofrontal cortex to action selection.
%B Annals of the New York Academy of Sciences
%D 2007
%C United States
%I New York Acad Sciences
%V 1121
%N 
%P 174-192
%@ 0077-8923
%X A number of recent findings suggest that the orbitofrontal cortex (OFC) influences action selection by providing information about the incentive value of behavioral goals or outcomes. However, much of this evidence has been derived from experiments using Pavlovian conditioning preparations of one form or another, making it difficult to determine whether the OFC is selectively involved in stimulus-outcome learning or whether it plays a more general role in processing reward value. Although many theorists have argued that these are fundamentally similar processes (i.e., that stimulus-reward learning provides the basis for choosing between actions based on anticipated reward value), several behavioral findings indicate that they are, in fact, dissociable. We have recently investigated the role of the OFC in the control of free operant lever pressing using tests that independently target the effect of stimulus-outcome learning and outcome devaluation on performance. We found that OFC lesions disrupted the tendency of Pavlovian cues to facilitate instrumental performance but left intact the suppressive effects of outcome devaluation. Rather than processing goal value, therefore, we hypothesize that the contribution of the OFC to goal-directed action is limited to encoding predictive stimulus-outcome relationships that can bias instrumental response selection.
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Wiltgen, Brian J
%A Law, Matthew
%A Ostlund, Sean
%A Mayford, Mark
%A Balleine, Bernard W
%T The influence of Pavlovian cues on instrumental performance is mediated by CaMKII activity in the striatum.
%B The European journal of neuroscience
%D 2007
%C United Kingdom
%I Blackwell Publishing Ltd
%V 25
%N 8
%P 2491-2497
%@ 0953-816X
%X Pavlovian cues associated with reward exert a powerful motivational influence on the performance of goal-directed actions. This motivational process depends critically on the ventral striatum, although little is known about the cellular and molecular mechanisms that mediate it. In the current experiments we examined the role of calcium calmodulin-dependent kinase II (CaMKII) by using transgenic mice that express a constitutively active form of this kinase. We found that controlled expression of active CaMKII in the striatum did not affect learning but did impair the motivation of goal-directed actions by Pavlovian cues associated with reward. Mutant mice learned to lever press for reward, remained sensitive to outcome devaluation and contingency degradation manipulations, and were able to acquire Pavlovian responses to cues paired with reward. However, Pavlovian cues were completely unable to motivate lever pressing in mutant mice. This was true even in mice trained with the CaMKII transgene turned off and then tested with it turned on. We were also able to suppress transgene expression in impaired mutants and fully restore the motivational effects of reward cues in these animals. Therefore, the current experiments demonstrate that normal CaMKII activity in the striatum is essential for the motivational effects of reward cues on goal-directed actions.
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Winterbauer, Neil E
%A Balleine, Bernard W
%T The influence of amphetamine on sensory and conditioned reinforcement: evidence for the re-selection hypothesis of dopamine function.
%B Frontiers in integrative neuroscience
%D 2007
%C Switzerland
%I Frontiers Research Foundation
%V 1
%N 
%P 1-8
%@ 1662-5145
%X In four experiments we assessed the effect of systemic amphetamine on the ability of a stimulus paired with reward and a stimulus that was not paired with reward to support instrumental conditioning; i.e., we trained rats to press two levers, one followed by a stimulus that had been trained in a predictive relationship with a food outcome and the other by a stimulus unpaired with that reward. Here we show, in general accord with predictions from the dopamine re-selection hypothesis [Redgrave and Gurney (2006). Nat. Rev. Neurosci. 7, 967-975], that systemic amphetamine greatly enhanced the performance of lever press responses that delivered a visual stimulus whether that stimulus had been paired with reward or not. In contrast, amphetamine had no effect on the performance of responses on an inactive lever that had no stimulus consequences. These results support the notion that dopaminergic activity serves to mark or tag actions associated with stimulus change for subsequent selection (or re-selection) and stand against the more specific suggestion that dopaminergic activity is solely related to the prediction of reward.
%Z FOR Codes: 110999


%0 Journal Article
%A Balleine, Bernard
%T The neural basis of choice and decision-making
%B Journal of Neuroscience
%D 2007
%C United States
%I Society for Neuroscience
%V 27
%N 
%P 8159-8160
%@ 0270-6474
%X 
%Z FOR Codes: 110999


%0 Journal Article
%~ PubMed
%A Balleine, Bernard W
%A Delgado, Mauricio R
%A Hikosaka, Okihide
%T The role of the dorsal striatum in reward and decision-making.
%B The Journal of neuroscience
%D 2007
%C United States
%I Society for Neuroscience
%V 27
%N 31
%P 8161-8165
%@ 1529-2401
%X Although the involvement in the striatum in the refinement and control of motor movement has long been recognized, recent description of discrete frontal corticobasal ganglia networks in a range of species has focused attention on the role particularly of the dorsal striatum in executive functions. Current evidence suggests that the dorsal striatum contributes directly to decision-making, especially to action selection and initiation, through the integration of sensorimotor, cognitive, and motivational/emotional information within specific corticostriatal circuits involving discrete regions of striatum. We review key evidence from recent studies in rodent, nonhuman primate, and human subjects.
%Z FOR Codes: 110999