Declarative Memory

This is going to be quite a long post because we've got four articles to read for this week's lecture. The topic is declarative memory.

Article: A unified framework for the functional organization of the medial temporal lobes and the phenomenology of episodic memory
Author: Charan Ranganath

• Abstract
• What is the role of the medial temporal lobe in recognition memory (I'll get to what that is in a second)?
• Evidence is consistent with the notion that MTL subregions differ in terms of the kind of information that they process and represent.
• These subregions support episodic memory by binding item and context information.
• My comments
• What kind of information are we talking about here (I'm asking this question in the most concrete sense, with 'kind' here meaning some physically realizable data structure, not 'Oh, it "represents" short-term memories')?
• How is this information processed (i.e., what (neuronal) procedures (computations) underlie such processes)?
• What and how is this information represented?
• Side rant
• I'm so annoyed by neuroscientists throwing the words information, process, representation around as if they actually mean something when they say it. I feel like it is used to cover up ignorance about whatever said person is talking about.
• Presumably, we're talking about encoding, but no one actually knows how the brain encodes sensory input (e.g., how does the brain transform light into the color red?). That is, there's no indication as to what the brain's encoding scheme is other than that it probably involves spike trains.
• What in the hell does it mean to bind item and context information?
• Are they being 'added' together in some sense? Multiplied? Composed?
• I'm going to go out on a limb and suggest that the author didn't really consider what binary operation that neurons are carrying out to cause this so-called binding to occur.
• History of Recognition Memory
• Milner and colleagues showed that extensive damage to the MTL impairs the formation of new memories for events, while sparing many other processes, such as motor skill learning.
• Research in monkeys corroborates this finding, especially on a delayed non-match to sample task with a long delay or large list of items to-be-remembered is given.
• More research suggested that the perirhinal cortex (PRc) plays a large role in recognition memory (same task as previous point)
• Deficits were found even with no delay
• Lesions of hippocampus proper (or damage to HC through the fornix) had very mild effects on recognition memory
• Similar effects found in rats and humans
• Two Component Models of the MTL
• Cohen et. al's Relational Memory Theory
• PRc and parahippocampal cortex (PHc) encode specific constituent elements of an event (items)
• Hippocampus encodes representations of the relationships of the items.
• Complementary Learning Systems (CLS)
• Hippocampus is specialized to rapidly encode new information
• Sparse, minimally overlapping representations that are well suited to encode specific episodes
• PHc
• Represents stimuli and events in terms of their constituent features; stimuli with overlapping representations have overlapping neural representations.
• Aggleton and Brown's Unnamed Model
• The hippocampus supports recall or recognition based on conscious recollection.
• HC and PHc differentially support different subjective experiences.
• What do these models predict about recognition memory?
• Relational Theory
• Relational information depends on HC, but PRc should be sufficient to support item recognition without relational information
• CLS
• Emphasizes the distinction in the processes that underlie recognition memory (familiarity vs. recollection).
• What is the degree of overlap in recognition memory signals elicited by studied items and unstudied lures?
• The cortical component of the model produces a signal that indexes the global match between a cue and all previously learned information due to broad overlapping stimulus representations.
• The hippocampal component produces a bimodal signal: some proportion of old items are indistinguishable from new items and other have very strong responses that are easily distinguishable.
• The HC should typically be more involved in recollection and the PHc more so in familiarity.
• BIC Model



Figure 1

• Lesion Evidence Suggests a Disproportionate Role for the Hippocampus in Recollection
• Familiarity-based recognition is supported by item representations in PRc
• Recollection should additionally depend on the HC and PHc to recover information about the corresponding study context.
• Humans with incidental fornix damage showed significant recollection impairments, whereas familiarity was intact.
• Similar data are seen in rats and humans (asymmetric ROC curve intact, symmetric when not)



Figure 2 (A) Humans (B) Rats

• Functional Imaging Evidence for Dissociable Roles of MTL Subregions in Item Recognition
• Familiarity is related to the strength of an item's representation
• Emerges as a byproduct of experience dependent tuning of the representation of an item during encoding.
• PRc activity should differ during encoding of items that will be recognized primarily on the basis of familiarity relative to items that will be missed, and PRc activity during retrieval should be sensitive to gradations in familiarity.
• Recollection depends on the successful encoding of the item and the association between the item and the context.
• Input from PRc to the hippocampus may trigger completion of the activity pattern that occurred during the learning event and lead to activation of the associated contextual representations in PHc networks. Finally, output from PHc to neocortical regions would elicit the reinstantiation of neocortical representations of the various aspects of the contextual state at the time of the original encoding event, thereby leading to recollection.
• Thus, hippocampal and PHc activity during encoding and retrieval should be higher for items that are subsequently recollected relative to items that are subsequently recognized primarily on the basis of familiarity.
• What does process-pure mean?
• PRc activation was increased during encoding of word pairs in the context of a definition (thereby encouraging their treatment as a single novel item or concept), as compared with encoding of word pairs in the context of a sentence frame (thereby encouraging them to be treated as separate items)
• The MTL is not the Site of Conscious Recollection
• MTL can support recovery of some kinds of information about the past, and this information does not always correspond to conscious experience. If this is the case, how do we make sense of the role of the hippocampus in conscious experiences like recollection? Although the availability of contextual information is generally a prerequisite for recollection, recollective experience ultimately is the outcome of a constructive process by which recovered information is used to make an attribution about the past.

Adaptive Coding of Reward Value by Dopamine Neurons

Authors: Philippe N. Tobler, Christopher Fiorillo, Wolfram Schultz

Summary: Midbrain DA neurons adapt to information provided by reward-predicting stimuli. The neuronal responses changed relative to the expected reward value; gain changed relative to the variance.

• Background:
• Expected Value (i.e., the 'average')

$$E\left[X\right]=\int_{-\infty}^{\infty}x\!\cdot \!p\left(x\right)\,\,\,\,\mathrm{d}x$$

• "In order to select the action associated with the largest reward, it is critical that the neural representation of reward has minimal uncertainty."
• I'm not sure this is in line with the basic tenets of information theory.
• I'll ignore this statement's inconsistency with information theory, what does it even mean? How does a 'representation' have an associated uncertainty?
• "... the representational capacity of the brain is limited, as exemplified by its finite number of neurons and the limited number of possible spike outputs of each neuron."
• Also a fundamentally inconsistent statement from what we know about computing machines.
• You don't need an infinite number of neurons to represent infinitely many things; you need procedures that can take an arbitrary input and produce an output from a possibly infinite set. What is finite is the encoding scheme used by a computing machine.
• An example is the operation of addition as implemented in a modern computer.
• Computers are not infinite in any sense of the word yet somehow they can do arbitrary precision arithmetic.
• As for the second statement, it's a little more plausible although I'm not sure anyone knows what the functional significance of spike outputs are. This is simply what we observe when record neuronal activity from the brain in response to a stimulus.
• Experiment(s):
• Five stimuli

• Each indicated the probability that a specific volume would be delivered 2 seconds after stimulus onset.
• Monkeys started to lick once they learned that the visual stimulus predicted a reward (A).
• Transient activation of DA neurons increased monotonically with the expected volume associated with each stimulus (B and C).
• Are individual neurons sensitive to probability and/or magnitude?
• Measured both magnitude and probability independently and found a correlation between the two (spikes / ml).
• When Tobler says the expected reward value does he really mean just he product of the probability and the magnitude?
• What is the extent to which DA neurons discriminate between different volumes of unpredicted liquid?
• How does DA neuron activity scale with the difference between actual and expected reward?

• Look at DA responses at the time of the reward from experiment shown in figure 1.
• 1A shows that animals can discriminate between stimuli.
• The larger of the two volumes always elicited an increase in activity at the time of the reward, and the smaller a decrease.
• The magnitude of activation or suppression appeared to be identical in each case.
• DA neurons do not scale according to the absolute difference between actual and expected reward.
• The gain of the neural responses appeared to adapt according to the discrepancy in volume between the two potential outcomes.
• Figure 4C to the right shows the median neural responses as a function of liquid volume and.
• Large 'difference' or 'variance' between expected reward magnitude and actual reward magnitude shows less activation small shows more.
• It doesn't matter what the absolute value of the difference between the smaller and larger rewards is, as long as they have an equal probability of occurrence.
• The larger of the two rewards always elicited the same increase and the smaller the same decrease regardless of absolute magnitude.
• Conclusions
• The authors suggest then, that activity in DA neurons carries information on the magnitude of reward.
• The intuitive notion is something like: "Adjust the animal's behavior via brain activity such that the reward outcomes that are most probable elicit the least variable response(s), regardless of the absolute size of the reward."

The Basolateral Amygdala Is Critical to Expression of Pavlovian and Instrumental Outcome-Specific Reinforcer Devaluation Effects

Authors: Alexander Johnson, Michela Gallagher, and Peter C. Holland

Journal: Journal of Neuroscience

Year: 2009

Keywords: amygdala, outcome representations, cues, incentive properties, reinforcement learning

• Summary
• The language used in (the abstract of) this paper is EXTREMELY confusing.
• Here's the gist: past research has examined the performance of BLA lesioned rats in devaluation procedures. It is clear that the BLA plays a role in the establishment of outcome representations that link cues to the incentive properties of reinforcers. The authors are asking what role the BLA plays once these outcome representations have been established.
• Two articles are cited in abstract:
• Pickens et. al (2003) found normal devaluation performance in rats when BLA lesions were made AFTER pavlovian light-food pairings but BEFORE devaluation by food-toxin pairings.
• Ostlund and Balleine (2008) found normal devaluation performance in rats when BLA lesions were MADE after instrumental training with MULTIPLE instrumental responses and food reinforcers but BEFORE devaluation of one reinforcer by selective satiation.
• They find here that when multiple reinforcers were used, POST-training BLA lesions disrupted the expression of devaluation performance in rats, using either pavlovian or instrumental training procedures and either conditioned taste aversion or satiation devaluation procedures.

• Introduction
• BLA damage shows impaired performance during reinforcer devaluation tasks.
• The value of the food reinforcer is reduced by satiation or food-toxin pairings after the completion of cue or response training.
• First Paradigm
• Animals are trained to associate either a neutral stimulus or a response with a particular reinforcer.
• AFTER training, the reinforcer is devalued by either motivational (e.g., prefeeding the reinforcer) or associative (e.g., pairing the reinforcer with illness)
• Finally, cue OR response performance is assessed, usually in the absence of the reinforcer.
• Normal animals show spontaneous reductions in performance, whereas animals with PREtraining BLA lesions typically do NOT.
• Pickens et al. (2003)
• Said BLA is required ONLY for acquisition of such outcome representations but NOT for maintaining them, modifying them, or using them to guide subsequent behavior. Found that rats lesioned AFTER conditioning but BEFORE devaluation of the food by food-illness pairings showed NORMAL devaluation effects.
• Ostlund and Balleine (2008) found that intact BLA function was required for integrating changes in reinforcer value with PREVIOUSLY acquired reinforcer representations to guide performance.
• Differences in the Paradigms
• Pickens et al. (2003)
• Associative conditioning
• Single reinforcer
• Taste aversion
• Ostlund and Balleine (2008)
• Instrumental conditioning
• Multiple reinforcers
• Selective satiation
• This paper's experiments are used to examine the role of training contingency and devaluation procedure in determining the effects of post-training BLA lesions on reinforcer devaluation performance in rats trained with multiple reinforcers
• Multiple outcome instrumental training
• Effects of devaluation by selective satiation
• Effects of devaluation by food-illness pairings
• Multiple outcome pavlovian training
• Effects of devaluation by selective satiation
• Effects of devaluation by food-illness pairings
• Material and Methods
• Materials
• Subjects
• Male, Long-Evans rats
• Surgeries
• NMDA lesions to LA in each hemisphere.
• N
• 1: 8, 8
• 2: 9, 10
• 3, 4: 8, 8
• Methods
• Experiment 1
• Food-cup training
• Rats food-deprived to 85% of body weight
• Preexposed for 2 h to each reinforcer
• orange or grape Kool-Aid
• 64-min food cup training session on each of 2 consecutive days
• Each session yielded 16 deliveries of a specific reinforcer and order of flavor presentation was counterbalanced.
• Instrumental training
• 2 instrumental training sessions per day separated by 2 h each
• 1 with left only 1 with right
• order alternated daily
• Response-outcome contingencies fully counterbalanced such that for half of the rats left lever responses resulted in delivery of grape and responses on the right lever produced delivery of orange, whereas the remaining rats were assigned the opposite contingencies
• First 3 days
• 30 min sessions in which each response was reinforced on fixed-interval schedule
• Then 20 min sessions and reinforcer delivery switched to random ratio schedule of reinforcement leading to 14 total sessions
• On average every 5 responses resulted in reinforcer delivery
• 3, 3, 3, 5: 5, 10, 15, 20
• number of sessions : average number of responses until reinforcement delivery
• 
  
• Instrumental reinforcer devaluation: sensory specific satiety, extinction, and choice test
• After surgery, rats were prefed with one of the two possible outcomes.
• Identity of the solution was counterbalanced across the previous response-outcome contingencies
• After 2 h rats were given a 20-min extinction test
• no reinforcements were given with responses
• except now both levers are available for responding
• What's the point?
• absence of the reinforcers ensures that test performance reflects an interaction of response-outcome information acquired during initial training with some internal representation of the status of the outcome as a goal after satiety treatment. To the extent that responding was controlled by the current value of the reinforcer anticipated after each of the two responses (left and right lever presses), rats would preferentially perform the response that had been reinforced previously with the reinforcer that had not been prefed (i.e., the non-devalued response).
• Effectiveness of the prefeeding devaluation treatment in altering the rats' preference for a reinforcer.
• Each rat given access to two drinking bottles in its home cage, one containing 25 ml of the prefed reinforcer and the other containing 25 ml of the other reinforcer
• You'd expect that consumption would be greater for the non-prefed reinforcer
• Experiment 2
• Instrumental training
• Same as experiment 1
• Instrumental reinforcer devaluation: conditioned taste aversion, extinction, and choice test
• Reinforcer paired with LiCl
• Days 1, 3, and 5
• All rats received 50 ml of the paired reinforcer for 15 min, followed by an injection of 0.3 M LiCl at 5 ml/kg
• Days 2, 4, and 6
• all rats received 50 ml of the unpaired reinforcer for 15 min
• Extinction test same as experiment 1
• To confirm the taste aversion readily transferred to the operant chambers, a 15 min consumption choice test was performed with rats given 25 ml of simultaneous access to both reinforcers in metal cups attached to the chamber floors.
• Experiment 3
• Pavlovian training
• Food-cup like experiment 1
• Two sessions per day ISI: 2 h
• 1 with 1500 kHz tone, 1 with white noise
• five 10s presentations of the stimulus, followed by delivery of 0.1 ml of either grape or orange solution, with a variable ITI that averaged 4 min.
• Rats received a total of 10 sessions of pavlovian training, order of sessions alternating daily
• After completion neurotoxic BLA surgeries were given, other half of rats were given sham lesions
• Pavlovian reinforcer devaluation: conditioned taste aversion, extinction, choice test
• After recovery experiment 2 taste aversion condition was given
• Pavlovian extinction test
• four 10s presentation of each stimulus tone and noise with a 4 min fixed interval between stimulus presentations
• Experiment 4
• Same as 3 except sensory-specific procedures that were the same as 1 were used to devalue on reinforcer before the extinction and reinforcer choice tests
• Results
• BLA lesions were large ~90% damage to lateral, basal, and accessory basal nuclei and 50% damage to ant and post basomedial nuclei.
• Experiment 1
• Instrumental training
• All rats displayed similar rates of responding for both reinforcers and increased their response rates after increments int he response-reinforcer schedule (as expected).
• No interactions
• Extinction test
• Sham-lesioned rats that were prefed resulted in a suppression of responding to the lever previously associated with that reinforcer compared with responding on the alternate (non-devalued) lever.
• BLA-lesioned rats displayed a small preference for the lever associated with the devalued reinforcer.
• Lesion-response interaction
• What exactly does this mean?
• Discussion
• Questions
• Why could any of the differences in the experimental paradigms have contributed to the different outcomes observed?

Title: Dopamine and cAMP-Regulated Phosphoprotein 32 kDa Controls Both Striatal Long-Term Depression and Long-Term Potentiation, Opposing Forms of Synaptic Plasticity

Authors: Paolo Calabresi, 

Paolo Gubellini, 

Diego Centonze, 

Barbara Picconi, 

Giorgio Bernardi, K

arima Chergui, 

Per Svenningsson, 

Allen A. Fienberg, and 

Paul Greengard

Journal: Journal of Neuroscience

Year: 2000

• Abstract
• The authors provide evidence that "the D1-like receptor-dependent activation of DA and cyclic adenosine 3',5' monophosphate-regulated phosphoprotein 32 kDa is a crucial step for the induction of both long-term depression (LTD) and long-term potentiation (LTP) ..."
• "Formation of LTD and LTP requires the activation of protein kinase G and protein kinase A in striatal projection neurons. These kinases appear to be stimulated by the activation of D1-like receptors in distinct neuronal populations."
• Introduction
• Facts about spiny neurons and the striatum
• "... both nigral DAergic inputs and cortical glutamatergic terminal converge on the same striatal neuronal subtype, the spiny projection neuron, which represents >90% of the striatal cell population and is the only cell type projecting out of the striatum.
• "Medium spiny neurons contain both D1-like (D1, D5) and D2-like (D2, D3, D4) DA receptors and also express both NMDA and non-NMDA classes of ionotropic glutamate receptors."
• "In the striatum, D1- and D2-like receptors trigger opposite effects on intracellular levels of cAMP, stimulating and inhibiting adenylyl cyclase activity."
• The activity of cAMP-dependent PKA is modulated by adenylyl cyclase activity.
• A major PKA substrate is cyclic adenosine 3',5' monophosphate-regulated phosphoprotein 32 kDa (DARPP-32)
• "DARPP-32 is expressed in very high concentrations in virtually all spiny neurons and acts (?), in its phosphorylated but not dephosphorylated form, as a potent inhibitor or protein phosphatase-1 (PP-1). PP-1 regulates the phosphorylation state and activity of many physiological effectors, including NMDA and AMPA glutamate receptors."
• Aim: "The aim of the present study was to address how the concomitant activation of ionotropic glutamate receptors and D1-like DA receptors initiates a cascade of biochemical events leading to the formation of opposing forms of corticostriatal plasticity, namely, LTD and LTP"
• Materials and Methods
• Electrophysiological Experiments
• Biological Experiments
• Results

• No significant difference in the resting membrane potential, input resistance, and current-voltage relationship in neurons recorded from the two groups of animals.
• AMPA receptor antagonist CNQX suppressed EPSPs in both sets of animals.
• Removal of Mg$^{2+}$ (removes the voltage dependent block of NMDA receptors) was needed to reveal an NMDA component of the EPSP that could be block by APV (?)
• High Frequency Stimulation
• Three spike trains of 100 Hz frequency, 3 s duration with 20 sec ISI of CORTICOSTRIATAL fibers produced 'long-term' chagnes in the AMPLITUDE of EPSPs in wild-type mice.
• This caused LDP in the presence of Mg$^{2+}$ and LTP in the absence of Mg$^{2+}$
• When knockout mice were given the same stimulation they show neither LTP nor LDP -> DARPP/PP-1 is necessary for AMPA's LT/DP related activity.
• Direct test of whether inability to inhibit PP-1 is what causes lack of plasticity
• Perform the same experiment and then bath the brain in okadaic acid and  calcyculin A (PP-1 inhibitors) 
• Okadaic acid restores both
• Calcyculin A restores only LTP
• No main effect of either on EPSPs
• Role of D1-like DA receptors and PKA in corticostriatal LTD and LTP
• Testing the blockade of D1-like DA receptors was effective in block both forms of learning.
• D1-like DA receptor antagonist SCH 23990 prevented LTD and LTP in wild-type mice.
• Interesting
• Next in the cascade is PKA induced phosphorylation of DARPP-32 so let's inhibit that

• Intracellular injection of H89 (PKA inhibitor) block LTP but not LTD in wild-type mice.
• However, when this was added to the Mg$^{2+}$ solution both forms of learning were prevented.
• Suggests different cellular loci for PKA action on learning
• For LTP but not for LTD this particular pathway is activated POST-synaptically on spiny neurons.
• More to come...
• Discussion