Declarative Memory

This is going to be quite a long post because we've got four articles to read for this week's lecture. The topic is declarative memory.

Article: A unified framework for the functional organization of the medial temporal lobes and the phenomenology of episodic memory
Author: Charan Ranganath

• Abstract
• What is the role of the medial temporal lobe in recognition memory (I'll get to what that is in a second)?
• Evidence is consistent with the notion that MTL subregions differ in terms of the kind of information that they process and represent.
• These subregions support episodic memory by binding item and context information.
• My comments
• What kind of information are we talking about here (I'm asking this question in the most concrete sense, with 'kind' here meaning some physically realizable data structure, not 'Oh, it "represents" short-term memories')?
• How is this information processed (i.e., what (neuronal) procedures (computations) underlie such processes)?
• What and how is this information represented?
• Side rant
• I'm so annoyed by neuroscientists throwing the words information, process, representation around as if they actually mean something when they say it. I feel like it is used to cover up ignorance about whatever said person is talking about.
• Presumably, we're talking about encoding, but no one actually knows how the brain encodes sensory input (e.g., how does the brain transform light into the color red?). That is, there's no indication as to what the brain's encoding scheme is other than that it probably involves spike trains.
• What in the hell does it mean to bind item and context information?
• Are they being 'added' together in some sense? Multiplied? Composed?
• I'm going to go out on a limb and suggest that the author didn't really consider what binary operation that neurons are carrying out to cause this so-called binding to occur.
• History of Recognition Memory
• Milner and colleagues showed that extensive damage to the MTL impairs the formation of new memories for events, while sparing many other processes, such as motor skill learning.
• Research in monkeys corroborates this finding, especially on a delayed non-match to sample task with a long delay or large list of items to-be-remembered is given.
• More research suggested that the perirhinal cortex (PRc) plays a large role in recognition memory (same task as previous point)
• Deficits were found even with no delay
• Lesions of hippocampus proper (or damage to HC through the fornix) had very mild effects on recognition memory
• Similar effects found in rats and humans
• Two Component Models of the MTL
• Cohen et. al's Relational Memory Theory
• PRc and parahippocampal cortex (PHc) encode specific constituent elements of an event (items)
• Hippocampus encodes representations of the relationships of the items.
• Complementary Learning Systems (CLS)
• Hippocampus is specialized to rapidly encode new information
• Sparse, minimally overlapping representations that are well suited to encode specific episodes
• PHc
• Represents stimuli and events in terms of their constituent features; stimuli with overlapping representations have overlapping neural representations.
• Aggleton and Brown's Unnamed Model
• The hippocampus supports recall or recognition based on conscious recollection.
• HC and PHc differentially support different subjective experiences.
• What do these models predict about recognition memory?
• Relational Theory
• Relational information depends on HC, but PRc should be sufficient to support item recognition without relational information
• CLS
• Emphasizes the distinction in the processes that underlie recognition memory (familiarity vs. recollection).
• What is the degree of overlap in recognition memory signals elicited by studied items and unstudied lures?
• The cortical component of the model produces a signal that indexes the global match between a cue and all previously learned information due to broad overlapping stimulus representations.
• The hippocampal component produces a bimodal signal: some proportion of old items are indistinguishable from new items and other have very strong responses that are easily distinguishable.
• The HC should typically be more involved in recollection and the PHc more so in familiarity.
• BIC Model



Figure 1

• Lesion Evidence Suggests a Disproportionate Role for the Hippocampus in Recollection
• Familiarity-based recognition is supported by item representations in PRc
• Recollection should additionally depend on the HC and PHc to recover information about the corresponding study context.
• Humans with incidental fornix damage showed significant recollection impairments, whereas familiarity was intact.
• Similar data are seen in rats and humans (asymmetric ROC curve intact, symmetric when not)



Figure 2 (A) Humans (B) Rats

• Functional Imaging Evidence for Dissociable Roles of MTL Subregions in Item Recognition
• Familiarity is related to the strength of an item's representation
• Emerges as a byproduct of experience dependent tuning of the representation of an item during encoding.
• PRc activity should differ during encoding of items that will be recognized primarily on the basis of familiarity relative to items that will be missed, and PRc activity during retrieval should be sensitive to gradations in familiarity.
• Recollection depends on the successful encoding of the item and the association between the item and the context.
• Input from PRc to the hippocampus may trigger completion of the activity pattern that occurred during the learning event and lead to activation of the associated contextual representations in PHc networks. Finally, output from PHc to neocortical regions would elicit the reinstantiation of neocortical representations of the various aspects of the contextual state at the time of the original encoding event, thereby leading to recollection.
• Thus, hippocampal and PHc activity during encoding and retrieval should be higher for items that are subsequently recollected relative to items that are subsequently recognized primarily on the basis of familiarity.
• What does process-pure mean?
• PRc activation was increased during encoding of word pairs in the context of a definition (thereby encouraging their treatment as a single novel item or concept), as compared with encoding of word pairs in the context of a sentence frame (thereby encouraging them to be treated as separate items)
• The MTL is not the Site of Conscious Recollection
• MTL can support recovery of some kinds of information about the past, and this information does not always correspond to conscious experience. If this is the case, how do we make sense of the role of the hippocampus in conscious experiences like recollection? Although the availability of contextual information is generally a prerequisite for recollection, recollective experience ultimately is the outcome of a constructive process by which recovered information is used to make an attribution about the past.

Adaptive Coding of Reward Value by Dopamine Neurons

Authors: Philippe N. Tobler, Christopher Fiorillo, Wolfram Schultz

Summary: Midbrain DA neurons adapt to information provided by reward-predicting stimuli. The neuronal responses changed relative to the expected reward value; gain changed relative to the variance.

• Background:
• Expected Value (i.e., the 'average')

$$E\left[X\right]=\int_{-\infty}^{\infty}x\!\cdot \!p\left(x\right)\,\,\,\,\mathrm{d}x$$

• "In order to select the action associated with the largest reward, it is critical that the neural representation of reward has minimal uncertainty."
• I'm not sure this is in line with the basic tenets of information theory.
• I'll ignore this statement's inconsistency with information theory, what does it even mean? How does a 'representation' have an associated uncertainty?
• "... the representational capacity of the brain is limited, as exemplified by its finite number of neurons and the limited number of possible spike outputs of each neuron."
• Also a fundamentally inconsistent statement from what we know about computing machines.
• You don't need an infinite number of neurons to represent infinitely many things; you need procedures that can take an arbitrary input and produce an output from a possibly infinite set. What is finite is the encoding scheme used by a computing machine.
• An example is the operation of addition as implemented in a modern computer.
• Computers are not infinite in any sense of the word yet somehow they can do arbitrary precision arithmetic.
• As for the second statement, it's a little more plausible although I'm not sure anyone knows what the functional significance of spike outputs are. This is simply what we observe when record neuronal activity from the brain in response to a stimulus.
• Experiment(s):
• Five stimuli

• Each indicated the probability that a specific volume would be delivered 2 seconds after stimulus onset.
• Monkeys started to lick once they learned that the visual stimulus predicted a reward (A).
• Transient activation of DA neurons increased monotonically with the expected volume associated with each stimulus (B and C).
• Are individual neurons sensitive to probability and/or magnitude?
• Measured both magnitude and probability independently and found a correlation between the two (spikes / ml).
• When Tobler says the expected reward value does he really mean just he product of the probability and the magnitude?
• What is the extent to which DA neurons discriminate between different volumes of unpredicted liquid?
• How does DA neuron activity scale with the difference between actual and expected reward?

• Look at DA responses at the time of the reward from experiment shown in figure 1.
• 1A shows that animals can discriminate between stimuli.
• The larger of the two volumes always elicited an increase in activity at the time of the reward, and the smaller a decrease.
• The magnitude of activation or suppression appeared to be identical in each case.
• DA neurons do not scale according to the absolute difference between actual and expected reward.
• The gain of the neural responses appeared to adapt according to the discrepancy in volume between the two potential outcomes.
• Figure 4C to the right shows the median neural responses as a function of liquid volume and.
• Large 'difference' or 'variance' between expected reward magnitude and actual reward magnitude shows less activation small shows more.
• It doesn't matter what the absolute value of the difference between the smaller and larger rewards is, as long as they have an equal probability of occurrence.
• The larger of the two rewards always elicited the same increase and the smaller the same decrease regardless of absolute magnitude.
• Conclusions
• The authors suggest then, that activity in DA neurons carries information on the magnitude of reward.
• The intuitive notion is something like: "Adjust the animal's behavior via brain activity such that the reward outcomes that are most probable elicit the least variable response(s), regardless of the absolute size of the reward."

The Basolateral Amygdala Is Critical to Expression of Pavlovian and Instrumental Outcome-Specific Reinforcer Devaluation Effects

Authors: Alexander Johnson, Michela Gallagher, and Peter C. Holland

Journal: Journal of Neuroscience

Year: 2009

Keywords: amygdala, outcome representations, cues, incentive properties, reinforcement learning

• Summary
• The language used in (the abstract of) this paper is EXTREMELY confusing.
• Here's the gist: past research has examined the performance of BLA lesioned rats in devaluation procedures. It is clear that the BLA plays a role in the establishment of outcome representations that link cues to the incentive properties of reinforcers. The authors are asking what role the BLA plays once these outcome representations have been established.
• Two articles are cited in abstract:
• Pickens et. al (2003) found normal devaluation performance in rats when BLA lesions were made AFTER pavlovian light-food pairings but BEFORE devaluation by food-toxin pairings.
• Ostlund and Balleine (2008) found normal devaluation performance in rats when BLA lesions were MADE after instrumental training with MULTIPLE instrumental responses and food reinforcers but BEFORE devaluation of one reinforcer by selective satiation.
• They find here that when multiple reinforcers were used, POST-training BLA lesions disrupted the expression of devaluation performance in rats, using either pavlovian or instrumental training procedures and either conditioned taste aversion or satiation devaluation procedures.

• Introduction
• BLA damage shows impaired performance during reinforcer devaluation tasks.
• The value of the food reinforcer is reduced by satiation or food-toxin pairings after the completion of cue or response training.
• First Paradigm
• Animals are trained to associate either a neutral stimulus or a response with a particular reinforcer.
• AFTER training, the reinforcer is devalued by either motivational (e.g., prefeeding the reinforcer) or associative (e.g., pairing the reinforcer with illness)
• Finally, cue OR response performance is assessed, usually in the absence of the reinforcer.
• Normal animals show spontaneous reductions in performance, whereas animals with PREtraining BLA lesions typically do NOT.
• Pickens et al. (2003)
• Said BLA is required ONLY for acquisition of such outcome representations but NOT for maintaining them, modifying them, or using them to guide subsequent behavior. Found that rats lesioned AFTER conditioning but BEFORE devaluation of the food by food-illness pairings showed NORMAL devaluation effects.
• Ostlund and Balleine (2008) found that intact BLA function was required for integrating changes in reinforcer value with PREVIOUSLY acquired reinforcer representations to guide performance.
• Differences in the Paradigms
• Pickens et al. (2003)
• Associative conditioning
• Single reinforcer
• Taste aversion
• Ostlund and Balleine (2008)
• Instrumental conditioning
• Multiple reinforcers
• Selective satiation
• This paper's experiments are used to examine the role of training contingency and devaluation procedure in determining the effects of post-training BLA lesions on reinforcer devaluation performance in rats trained with multiple reinforcers
• Multiple outcome instrumental training
• Effects of devaluation by selective satiation
• Effects of devaluation by food-illness pairings
• Multiple outcome pavlovian training
• Effects of devaluation by selective satiation
• Effects of devaluation by food-illness pairings
• Material and Methods
• Materials
• Subjects
• Male, Long-Evans rats
• Surgeries
• NMDA lesions to LA in each hemisphere.
• N
• 1: 8, 8
• 2: 9, 10
• 3, 4: 8, 8
• Methods
• Experiment 1
• Food-cup training
• Rats food-deprived to 85% of body weight
• Preexposed for 2 h to each reinforcer
• orange or grape Kool-Aid
• 64-min food cup training session on each of 2 consecutive days
• Each session yielded 16 deliveries of a specific reinforcer and order of flavor presentation was counterbalanced.
• Instrumental training
• 2 instrumental training sessions per day separated by 2 h each
• 1 with left only 1 with right
• order alternated daily
• Response-outcome contingencies fully counterbalanced such that for half of the rats left lever responses resulted in delivery of grape and responses on the right lever produced delivery of orange, whereas the remaining rats were assigned the opposite contingencies
• First 3 days
• 30 min sessions in which each response was reinforced on fixed-interval schedule
• Then 20 min sessions and reinforcer delivery switched to random ratio schedule of reinforcement leading to 14 total sessions
• On average every 5 responses resulted in reinforcer delivery
• 3, 3, 3, 5: 5, 10, 15, 20
• number of sessions : average number of responses until reinforcement delivery
• 
  
• Instrumental reinforcer devaluation: sensory specific satiety, extinction, and choice test
• After surgery, rats were prefed with one of the two possible outcomes.
• Identity of the solution was counterbalanced across the previous response-outcome contingencies
• After 2 h rats were given a 20-min extinction test
• no reinforcements were given with responses
• except now both levers are available for responding
• What's the point?
• absence of the reinforcers ensures that test performance reflects an interaction of response-outcome information acquired during initial training with some internal representation of the status of the outcome as a goal after satiety treatment. To the extent that responding was controlled by the current value of the reinforcer anticipated after each of the two responses (left and right lever presses), rats would preferentially perform the response that had been reinforced previously with the reinforcer that had not been prefed (i.e., the non-devalued response).
• Effectiveness of the prefeeding devaluation treatment in altering the rats' preference for a reinforcer.
• Each rat given access to two drinking bottles in its home cage, one containing 25 ml of the prefed reinforcer and the other containing 25 ml of the other reinforcer
• You'd expect that consumption would be greater for the non-prefed reinforcer
• Experiment 2
• Instrumental training
• Same as experiment 1
• Instrumental reinforcer devaluation: conditioned taste aversion, extinction, and choice test
• Reinforcer paired with LiCl
• Days 1, 3, and 5
• All rats received 50 ml of the paired reinforcer for 15 min, followed by an injection of 0.3 M LiCl at 5 ml/kg
• Days 2, 4, and 6
• all rats received 50 ml of the unpaired reinforcer for 15 min
• Extinction test same as experiment 1
• To confirm the taste aversion readily transferred to the operant chambers, a 15 min consumption choice test was performed with rats given 25 ml of simultaneous access to both reinforcers in metal cups attached to the chamber floors.
• Experiment 3
• Pavlovian training
• Food-cup like experiment 1
• Two sessions per day ISI: 2 h
• 1 with 1500 kHz tone, 1 with white noise
• five 10s presentations of the stimulus, followed by delivery of 0.1 ml of either grape or orange solution, with a variable ITI that averaged 4 min.
• Rats received a total of 10 sessions of pavlovian training, order of sessions alternating daily
• After completion neurotoxic BLA surgeries were given, other half of rats were given sham lesions
• Pavlovian reinforcer devaluation: conditioned taste aversion, extinction, choice test
• After recovery experiment 2 taste aversion condition was given
• Pavlovian extinction test
• four 10s presentation of each stimulus tone and noise with a 4 min fixed interval between stimulus presentations
• Experiment 4
• Same as 3 except sensory-specific procedures that were the same as 1 were used to devalue on reinforcer before the extinction and reinforcer choice tests
• Results
• BLA lesions were large ~90% damage to lateral, basal, and accessory basal nuclei and 50% damage to ant and post basomedial nuclei.
• Experiment 1
• Instrumental training
• All rats displayed similar rates of responding for both reinforcers and increased their response rates after increments int he response-reinforcer schedule (as expected).
• No interactions
• Extinction test
• Sham-lesioned rats that were prefed resulted in a suppression of responding to the lever previously associated with that reinforcer compared with responding on the alternate (non-devalued) lever.
• BLA-lesioned rats displayed a small preference for the lever associated with the devalued reinforcer.
• Lesion-response interaction
• What exactly does this mean?
• Discussion
• Questions
• Why could any of the differences in the experimental paradigms have contributed to the different outcomes observed?

Title: Dopamine and cAMP-Regulated Phosphoprotein 32 kDa Controls Both Striatal Long-Term Depression and Long-Term Potentiation, Opposing Forms of Synaptic Plasticity

Authors: Paolo Calabresi, 

Paolo Gubellini, 

Diego Centonze, 

Barbara Picconi, 

Giorgio Bernardi, K

arima Chergui, 

Per Svenningsson, 

Allen A. Fienberg, and 

Paul Greengard

Journal: Journal of Neuroscience

Year: 2000

• Abstract
• The authors provide evidence that "the D1-like receptor-dependent activation of DA and cyclic adenosine 3',5' monophosphate-regulated phosphoprotein 32 kDa is a crucial step for the induction of both long-term depression (LTD) and long-term potentiation (LTP) ..."
• "Formation of LTD and LTP requires the activation of protein kinase G and protein kinase A in striatal projection neurons. These kinases appear to be stimulated by the activation of D1-like receptors in distinct neuronal populations."
• Introduction
• Facts about spiny neurons and the striatum
• "... both nigral DAergic inputs and cortical glutamatergic terminal converge on the same striatal neuronal subtype, the spiny projection neuron, which represents >90% of the striatal cell population and is the only cell type projecting out of the striatum.
• "Medium spiny neurons contain both D1-like (D1, D5) and D2-like (D2, D3, D4) DA receptors and also express both NMDA and non-NMDA classes of ionotropic glutamate receptors."
• "In the striatum, D1- and D2-like receptors trigger opposite effects on intracellular levels of cAMP, stimulating and inhibiting adenylyl cyclase activity."
• The activity of cAMP-dependent PKA is modulated by adenylyl cyclase activity.
• A major PKA substrate is cyclic adenosine 3',5' monophosphate-regulated phosphoprotein 32 kDa (DARPP-32)
• "DARPP-32 is expressed in very high concentrations in virtually all spiny neurons and acts (?), in its phosphorylated but not dephosphorylated form, as a potent inhibitor or protein phosphatase-1 (PP-1). PP-1 regulates the phosphorylation state and activity of many physiological effectors, including NMDA and AMPA glutamate receptors."
• Aim: "The aim of the present study was to address how the concomitant activation of ionotropic glutamate receptors and D1-like DA receptors initiates a cascade of biochemical events leading to the formation of opposing forms of corticostriatal plasticity, namely, LTD and LTP"
• Materials and Methods
• Electrophysiological Experiments
• Biological Experiments
• Results

• No significant difference in the resting membrane potential, input resistance, and current-voltage relationship in neurons recorded from the two groups of animals.
• AMPA receptor antagonist CNQX suppressed EPSPs in both sets of animals.
• Removal of Mg$^{2+}$ (removes the voltage dependent block of NMDA receptors) was needed to reveal an NMDA component of the EPSP that could be block by APV (?)
• High Frequency Stimulation
• Three spike trains of 100 Hz frequency, 3 s duration with 20 sec ISI of CORTICOSTRIATAL fibers produced 'long-term' chagnes in the AMPLITUDE of EPSPs in wild-type mice.
• This caused LDP in the presence of Mg$^{2+}$ and LTP in the absence of Mg$^{2+}$
• When knockout mice were given the same stimulation they show neither LTP nor LDP -> DARPP/PP-1 is necessary for AMPA's LT/DP related activity.
• Direct test of whether inability to inhibit PP-1 is what causes lack of plasticity
• Perform the same experiment and then bath the brain in okadaic acid and  calcyculin A (PP-1 inhibitors) 
• Okadaic acid restores both
• Calcyculin A restores only LTP
• No main effect of either on EPSPs
• Role of D1-like DA receptors and PKA in corticostriatal LTD and LTP
• Testing the blockade of D1-like DA receptors was effective in block both forms of learning.
• D1-like DA receptor antagonist SCH 23990 prevented LTD and LTP in wild-type mice.
• Interesting
• Next in the cascade is PKA induced phosphorylation of DARPP-32 so let's inhibit that

• Intracellular injection of H89 (PKA inhibitor) block LTP but not LTD in wild-type mice.
• However, when this was added to the Mg$^{2+}$ solution both forms of learning were prevented.
• Suggests different cellular loci for PKA action on learning
• For LTP but not for LTD this particular pathway is activated POST-synaptically on spiny neurons.
• More to come...
• Discussion

Title: Dopamine gating of glutamatergic sensorimotor and incentive motivational input signals to the striatum

Author: Jon C. Horvitz

Journal: Behavioural Brain Research
Year: 2002

• Overview
• Traditional assumption: REWARD STIMULI are unique in producing a PHASIC dopamine (DA) response; other stimuli produce much more gradual (necessarily TONIC?) DA responses.
• Not assumed in this article
• Forebrain DA activity signals neither the "unpleasantness nor the pleasantness of the event" that elicited it.
• KEY IDEA: "Activation of substantia nigra (SN) and ventral tegmental area (VTA) DA neurons, and consequent release of nigrostriatal and mesolimbic DA to dorsal and ventral striatal target regions, modulates the processing of concurrent glutamate (Glu) inputs. [This] occurs under conditions of unexpected environmental change."
• Main question: How does DA transmission in the dorsal and ventral striatum influence behavior?
• What biological/environmental conditions lead to elevated DA activity?
• What information is carried by Glu inputs to striatal neurons?
• How are these Glu inputs modulated by DA activity?
• What is the ventral striatum?
• Nucleus accumbens
• Caudate-putamen
• Olfactory tubercle
• Here the ventral striatum means nucleus accumbens.
• Instead of thinking of DA as a 'chemical code' for reward, here it is argued that "environmentally elicited elevations in mesolimbic and nigrostriatal DA activity gate the input of reward signals to the striatum, just as they do for sensorimotor signals [...]"
• Such reward signals are likely to originate in orbitofrontal cortex and basolateral amygdala.
• What this means is that DA 'informs' striatal cells that an unexpected, important event has occurred.
• DA neurons respond to salient unexpected events
• It is unlikely that heightened attentional states are associated with a phasic DA response, because attentional systems would likely be recruited by both the presence of an unexpected event and the absence of an expected event. DA neurons increase activity only to the former, while they are inhibited by the latter.
• Time Course
• Very rapid onset
• Phasic activation increase in approximately 50-100 ms after stimulus onset
• It can't only be REWARD prediction error because of evidence showing that DA in prefrontal cortex (PFC) and nucleus accumbens (NA) is elevated under appetitive and aversive conditions.
• Primary Aim: "[P]rovide a framework that accounts both for the promiscuous DA response to salient events and for the large body of evidence showing that DA disruptions attentuate the impact of rewards (and punishers) on several aspects of behavior and learning."
• Important: DA may be considered a gatekeeper of glutamatergic information flow to the striatum.
• DA selectively promotes the processing of strong glutamate inputs to the striatum
• It turns out that DA acting at the D1 receptor increases the activity of glutamate at NMDA receptor sites. D2 however, does the opposite and reduces activity of glutamate at non-NMDA receptor sites.
• DA activity gates the throughput of sensorimotor and incentive motivational inputs to the striatum
• Striatal neurons receive inputs from a wide variety of 'types' of neurons. For example, the caudate, putamen, and ventral striatum contain neurons that respond to arm movements but only when the animal expects to receive a food reward following the movement. Conversely, very little activity is seen if a sound is expected to follow the movement.
• Striatal neurons respond to very abstract types of information.
• Ah, logic is so beautiful: [Individual striatal neurons] ... appear to be capable of representing the conjunction of two conditions[!]
• Striatal neurons are the (and CONDITIONS ...) Lisp function!
• What happens when nigrostriatal DA transmission is reduced?
• Not surprisingly, input signals are less likely to produce a response.
• Interestingly, when DA is depleted the striatal cell response is diminished but only in background activity NOT phasic activity. When DA is increased the phasic response is stronger than baseline and the background activity is actually reduced!
• There's evidence that suggests that DA modulates both the striatal response to current glutamate inputs and log-term changes in synaptic strength of these inputs.
• Striatal plasticity: stimulus-response learning, salience assignment to synaptic inputs, and/or stimulus-response-outcome chunking
• Schultz thinks that DA signals provide the 'difference' between reward occurrence and reward prediction. "The phasic DA response increases the synaptic strength betwee ncurrently ative striatal input and output elements, increasing the future likelihood that the current set of corticostriatal inputs (reward) will activate striatal outputs (motor responses)." The problem with this hypothesis is that if DA additionally responds to aversive events then modulation of behavior via this mechanism would mean that behavioral responses leading to aversive outcomes would be increased in number. So, says JH, DA's primary function will not likely be one involving the strengthening of stimulus-response (S-R) connections.
• So what does DA actually do then?
• Well, maybe strengthening of corticostriatal inputs via DA serves to increase the future salience of an event (or series of events).
• According to JCH this implies that additional processing of information is performed later in more specialized regions of basal ganglia. This would then imply that something downstream of the striatum is what is determining the reward value of a stimulus; it isn't indicated by the phasic DA response. He proposes frontal regions as the area that performs response selection.
• It's an indicator for salient environmental change
• Biologically this means that, due to the processes mentioned earlier, this change will cause an increase in "glutamate signal-to-noise ratios within dorsal and ventral striatal target sites"
• Thoughts and Questions
• What does it mean when 'a representation converges' onto a neuron?
• How do non-physical things (representations) converge onto physical things (neurons)?

Testing out Google's code prettifier...

% a MATLAB function
function x = some_func()
fs = 1000;
d = 1;
t = 0:1 / fs:d;
x = sin(t);
ex = 0.1 * randn(size(x)) + x;
end

class Point(object):
    """
    """

    def __init__(self, x, y):
        """__init__

        Arguments:
        - `x`:
        - `y`:
        """
        self._x = x
        self._y = y

    @property
    def x(self):
        return self._x

    @x.setter
    def x(self, value):
        self._x = value

    @x.deleter
    def x(self):
        del self._x

The Relationship Between Moments and Cumulants

The latest assignment in my biomedical signal processing class, followed by my modest attempt at the proof:

Show that
\begin{eqnarray*}
  c_{1} &=& m_{1}=\mu \\
  c_{2} &=& m_{2} - m_{1}^{2}=\sigma^{2}.
\end{eqnarray*}
That is, show that the first and second cumulants of a distribution are equal to the mean ($\mu$) and variance ($\sigma^{2}$), respectively.

Proof:
The $n$th moment of a continuous distribution, $m_{n}$ is defined as:
\begin{eqnarray*}
  m_{n} &=& E[x^{n}]\\
  &=& \int_{-\infty}^{\infty}x^{n}p\left(x\right)\,dx.
\end{eqnarray*}
A function that generates any moment is called a moment-generating function and is defined as $$E[e^{tX}].$$ where $X$ is a random variable and $E[f\left(X\right)]$ is the expected value of a function, $f$, of a random variable $X$.

If we expand the definition of $E[e^{tX}]$ we can see that
\begin{eqnarray*}
  E[e^{tX}] &=&
  \int_{-\infty}^{\infty}p\left(X\right)\left(\sum_{n=0}^{\infty}\frac{t^{n}X^{n}}{n!}\right)\,\,\mathrm{d}x\\
  &=& \int_{-\infty}^{\infty}p\left(X\right)\left(1 + tX + \frac{t^{2}X^{2}}{2!} + \cdots +
    \frac{t^{n}X^{n}}{n!}\right)\,\,\mathrm{d}x\\
  &=& 1 + tm_{1} + \frac{t^{2}m_{2}}{2!} + \cdots + \frac{t^{n}m_{n}}{n!}.
\end{eqnarray*}

If we evaluate the $n$th derivative of $E[e^{tX}]$ with respect to $t$ at $t=0$, that is, $$\left(\frac{\mathrm{d}^{n}}{\mathrm{d}t^{n}}E[e^{tX}]\right)_{t=0} =
\frac{\mathrm{d}^{n}}{\mathrm{d}t^{n}}\left(m_{0} + tm_{1} +
  \frac{t^{2}m_{2}}{2!} + \cdots + \frac{t^{n}m_{n}}{n!}\right)$$ we get the $n$th moment of a probability distribution.

The $n$th cumulant, $c_{n}$, of a continuous distribution is defined as:
$$\ln\left(E[e^{tX}]\right).$$
Since the power series definition of the natural logarithm of $x$ is
$$\ln\left(x\right) = \sum_{n=1}^{\infty}\frac{\left(-1\right)^{n+1}}{n}\left(x - 1\right)^{n},$$
we can rewrite the definition of the $n$th cumulant, $c_{n}$, as
\begin{eqnarray*}
  \ln\left(E[e^{tX}]\right) &=&
  \sum_{n=1}^{\infty}\frac{\left(-1\right)^{n+1}}{n}\left(tm_{1} +
    \frac{t^{2}m_{2}}{2!} + \cdots\right)^{n}
\end{eqnarray*}
Expanding the terms from the previous equation we see that,
\begin{eqnarray*}
  \ln\left(E[e^{tX}]\right) &=& \left(tm_{1} +
    \frac{t^{2}m_{2}}{2!} + \cdots\right) - \frac{1}{2}\left(tm_{1} +
    \frac{t^{2}m_{2}}{2!} + \cdots\right)^{2} + \cdots\\
&=&\left(tm_{1} +
    \frac{t^{2}m_{2}}{2!} + \cdots\right)-\frac{1}{2}\left(t^{2}m_{1}^{2}+\frac{t^{3}m_{1}m_{2}}{2}+\cdots\right) + \cdots.
\end{eqnarray*}
Now do a little algebra on the second term of $\ln\left(E[e^{tX}]\right)$ to get
$$\left(tm_{1}+\frac{t^{2}m_{2}}{2!}+\cdots\right)-\frac{t^{2}m_{1}^{2}}{2}-\frac{t^{3}m_{1}m_{2}}{4}+\cdots$$

The first derivative, with respect to $t$, evaluated at 0 gives $m_{1}$. The second derivative evaluated at 0 gives

$$\frac{\mathrm{d}^{2}}{\mathrm{d}t^{2}}\left(\frac{t^{2}m_{2}}{2!}\right) =m_{2}.$$
Then we see that $m_{2} - m_{1}^{2}$ gives the variance.

Thus it is shown that the first two cumulants of a distribution are the mean and variance, respectively.

Quicksort

I finally understand the quicksort algorithm. Python helped me with this. Here's how it works:

import random

def quicksort(lst):
    """quicksort
    """
    n = len(lst)
     
    # if the list has 1 or no elements
    if n < 2:
        return lst
     
    # remove a random element and make it the pivot value
    pivot = lst.pop(random.randint(0, n - 1))
     
    # elements less than pivot
    less = filter(lambda x: x < pivot, lst)
     
    # elements not less than pivot
    greater = filter(lambda x: not x < pivot, lst)
     
    # recursively sort the items
    return quicksort(less) + [pivot] + quicksort(greater)
 
if __name__ == '__main__':
    lst = range(10)
    random.shuffle(lst)
    print 'unsorted: {lst}'.format(lst=lst)
    print 'sorted: {lst}'.format(lst=quicksort(lst))

This is the algorithm implemented in Python using recursion. With recursion it is important to identify a base case to avoid infinite loops and stack overflows. In this case our base case is 1 or fewer elements because both a list of 1 element and the empty list are both already sorted.

Now, if you uncomment the print statement and run the code you can see how it works.

Step 1:

Pick a random element from the input list. Let lst = [9, 4, 2, 8, 7, 6, 5, 3, 1, 10], let pivot = 5.

Step 2:

Partition all of the elements less than pivot into a list less. Do the same for the elements that are greater than pivot and call it greater.

Example:

less = [4, 2, 3, 1], greater = [9, 8, 7, 6, 10]

This results in the function returning quicksort([4, 2, 3, 1]) + [5] + quicksort([9, 8, 7, 6, 10])

So now we have [..., 5, ...]

Step 3:

Repeat steps 1 and 2 until you reach your base case, that is, there is either 0 or 1 elements in less and greater

Continuing the previous example:

The next iteration of the recursive call results in quicksort([]) + [1] + quicksort([4, 2, 3]).

Now we have [1, ..., 5, ... ]

Next iteration gives quicksort([]) + [2] + quicksort([4, 3])

The updated list is: [1, 2, ..., 5, ...]

Next iteration: quicksort([]) + [3] + quicksort([4])

Updated list: [1, 2, 3, 4, 5, ....]

Great! We've sorted half the list. Now going to the greater half...

Next iteration: quicksort([8, 7, 6]) + [9] + quicksort([10])

Updated list: [1, 2, 3, 4, 5, ..., 9, 10]

Next iteration: quicksort([]) + [6] + quicksort([8, 7])

Updated list: [1, 2, 3, 4, 5, 6, ..., 9, 10]

Next iteration: quicksort([]) + [7] + quicksort([8])

Updated list: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Whoo hoo! We've sorted this (rather boring) list! I hope this has been informative.

TMS Perturbs Saccade Trajectories and Unmasks an Internal Feedback Controller for Saccades

Author(s): Minnan Xu-Wilson, Jing Tian, Reza Shadmehr, David S. Zee

Summary: Transcranial magnetic stimulation (TMS) centrally perturbs saccades and these saccades are corrected for the perturbation within the movement providing further support for internal feedback control of saccadic eye movements.

Question(s): Can we observe online correction of saccadic eye movements after a central perturbation via TMS?

Abbreviations: Postinhibitory Rebound (PIR)

• Introduction
• Saccades are known to be highly variable beasts.
• However, they generally arrive on target (+- 1.0 deg)
• Motor commands may depending on any or all of the following, but are not limited to:
• Stimulus content
• Stimulus predictability
• Whether or not the stimulus covaries with the target of a reaching movement
• Motivation
• While arm movements can be perturbed via a force-field, it has been difficult to perform an analogous procedure on eye movements.
• Current Methods (and their drawbacks)
• Facial Stimulation
• Painful
• Loud Noise
• Uncomfortable
• Shows habituation
• Let's use TMS!
• Preliminary findings
• "When a single pulse is applied immediately before or during a saccade, it engages a startle-like neural reflex that briefly alters the ongoing oculomotor commands, slowing or even transiently stopping the eye movement."
• "Despite this perturbation, the movement is corrected with commands that arrive later in the same saccade, accurately steering the eyes close to the target even when the target stimulus is no longer visible."
• Materials and Methods
• n = 5, 3 M
• Visually guided saccades
• Materials
• Saccades
• Bite-bar (annoying)
• Scleral search coil (probably painful)
• f_s = 1000 Hz
• Filters
• Low-pass 90 Hz Butterworth on eye position signals
• 3rd order Savitzky-Golay applied to position signals to derive velocity and acceleration signals
• 0.2 deg red laser beam projected 1 meter away from subjects
• Used 500 Hz frame rate video and an EyeLink 1000 to corroborate findings from search coils
• TMS
• 2.2 Tesla
• Stimulation strength: 50% - 60%
• Stimulated Cz (using EEG 10-20 system coordinates)
• Methods
• 16 deg/s criteria for saccades
• Exclusion criteria
1. Amp < 67% of target displacement
2. 100 ms > saccade reaction time > 500 ms
3. Abnormal trajectories due to blinks
• Pause criteria
• 2 clear peaks in velocity profile
• Local minima < 50 deg/s
• Experiment 1: Saccade Onset
• Fixate for 1500-2300 ms
• TMS triggered when 30 deg/s velocity threshold reached
• happened only on 67% of trials (determined probabilistically)
• Confound? Only three out of five subjects were tested to determine whether the sound alone elicited a peturbation. Why not all five when in the introduction they state that this is a commonly used method to achieve the same end goal as their experiment?
• Experiment 2: During the Saccade
• Determine how timing of TMS affects saccade trajectories
• Used oblique saccades because these would provide a larger time window for TMS to take effect.
• Given on 70% of trials
• at 5, 15, 25, 35, 45 and 55 ms after saccade onset (i.e., velocity threshold reached).
• Experiment 3: Before the saccade
• Assumed 180 ms saccade latency
• Triggered TMS w.r.t. target onset randomly at 40, 60, or 80 ms before the expected saccade onset.
• Analysis:
• Grouped trials into bins according to the actual time of TMS before saccade onset.
• Modeling
• Ramat et al., 2005
• Components
• "Two coupled excitatory burst neuron (EBN) and inhibitory burst neuron (IBN) pairs..."
• "Burst neurons fired at a rate that depends on the size of hte difference between the current estimate of eye position and the target position..."
• "Motor error calculated by integrating the velocity output from the burst neurons and then subtracting this estimate of current eye position from the desired goal of the movement--the integration served as a state estimator, providing an ongoing internal feedback to the system..."
• "The burst neurons' membranes were modeled as high-pass filters with adaptation"
• Results

• Experiment 1
• In 74% of trials the TMS perturbed the saccade trajectory regardless of where the brain was stimulated
• Always in the form of a pause in the velocity profile
• Effected persisted for 32 ms before the eyes reaccelerated
• No difference in slowing time of vertical and horizontal components of oblique saccades.
• No habituation over sets of trials (a friend pointed out to me that this is like asking whether there was habituation of curare, but the authors reported the statistics used to test this effect so it seems like it's relevant somehow...)
• Paused Saccades
• The size of the compensatory movement was highly correlated with the remaining distance to the target.
• Visual condition (whether target was blanked on saccade) made NO difference in the time it took for resumed movement to start nor the quality of compensation for error during the pause
• Interestingly, the final amplitude of a perturbed saccade was, on average, larger than a control saccade by 0.83
• Oddities
• No-pause saccades (26%) elicited the following 2 properties:
• Their amplitudes were generally smaller than control saccades by 0.84 degrees
• Amplitudes were generally l.10 deg larger than the initial amplitudes of saccades that paused and resumed
• Anti-saccades were tested because the effect may have been due to the involuntary nature of so-called "pro" saccades
• During horizontal saccades no effect of eyelid perturbation was found on eye trajectory, thus the perturbation in the eye was caused by TMS not some interaction between the TMS and the eyelid.
• Lid saccade pauses and eye saccade pauses are highly correlated
• No evidence for head motion causing the observed effect
• Experiment 2
• TMS applied late in the time course of a saccade could stop the saccade altogether

Why are so few subjects used in this line of research? There are only 5 subjects in the study and most saccade studies do not have any more than that. In an experiment that a colleague of mine and I are working on we have 5-6 subjects per condition and that still seems low to me.

Interestingly, these guys found that saccades that were perturbed were slower yet hypermetric when compared with non TMS'ed saccades. They account for this by suggesting that during the pause period, the burst neurons are inhibited by reactivated OPNs, leading to greater firing rate of burst neurons after the pause period. This effect was not greater than the firing rates seen after saccades of the same size were made after a normal fixation. I don't really understand how the "inherent delay in the feedback loop does not allow for complete compensation of this overshoot" can account for the paradox of hypermetric saccades with smaller peak amplitudes than saccades not perturbed by TMS.

Phillip: Why is habituation not important?

Nate: It's possible that the authors are conflating the notion of stimulated the nervous system with TMS versus a stimulus evoking a response in the nervous system. Hmmm let's talk about this tomorrow.

Three Programming Languages Compared

Let's take a look at three different programming languages--MATLAB, Python, and LUSH--and compare the ease of plotting with each as measured by the number of commands required to generate a simple sine wave on the interval [0, 2π] and save it to some relatively portable image format.



MATLAB:

t = 0:0.001:(2 * pi);
x = sin(t);
plot(t, x);
axis tight;
xlabel('$t$', 'interpreter', 'latex');
ylabel('$\sin(t)$', 'interpreter', 'latex');
title('$\sin(t)$', 'interpreter', 'latex');
legend({'$\sin(t)$'}, 'interpreter', 'latex');
set(gcf, 'PaperOrientation', 'landscape', 'PaperPosition', [0 0 11.5 8], ...
    'PaperSize', [11.5 8]);
saveas('temp', 'png');

Python using the IPython Shell:

t = arange(0, 2 * pi, 0.001)
x = sin(t)
plot(t, x)
xlabel('$t$')
ylabel('$\sin(t)$')
title('$\sin(t)$')
legend(['$\sin(t)$'])
savefig('temp', format='png')

LUSH:
(libload "libplot/plotter")
(defvar p (new Plotter))
(let ((w (ps-window "/home/cpcloud/Desktop/temp.ps")))
  (==> p PlotFunc "sin(t)" sin 0 +2pi+ 0.001 (alloccolor 0 0 1))
  (==> p SetXScale 0 +2pi+)
  (==> p SetYScale (- 1) 1)
  (==> p SetXLabel "t")
  (==> p SetYLabel "sin(t)")
  (==> p SetLegend "sin(t)")
  (==> p SetTitle "sin(t)")
  (==> p Redisplay))


;; not strictly necessary--it outputs a PS file without this which is perfectly fine
(sh "convert -density 150 -geometry 100% temp.png")

Clearly the winner here using the metric mentioned above is Python using the IPython shell. MATLAB  and LUSH are tied for second, however you might change your mind after seeing their respective plots.

Sadly, the plot created by LUSH leaves much to be desired and sends me running back to matplotlib and (GASP!) even MATLAB. LUSH's plotter isn't buggy per se, it's just not as refined as Python's or MATLAB's.

For some reason though, I still like programming in LUSH much more than programming in MATLAB. Lush is WAAAAY faster during matrix computations and the anonymous function syntax is much more forgiving, e.g., (lambda (x) (somefunction x)) versus @(x) somefunction(x). The ability to write C/C++ code mixed with LUSH code certainly doesn't hurt its coolness factor either.

I have to admit, though, when I took Yann LeCun's machine learning course at NYU I thought Lush was a crazy little language and all those parentheses were going to drive me insane!! Little did I know I would become slightly obsessed with it just under a year later...

Check out LUSH, it has a lot of potential!

* Next up: the LUSH code to make its plotting facility much more user-friendly. Stay tuned.

Dopamine and Learned Food Preferences

Authors: Anthony Sclafani, Khalid Touzani, and Richard J. Bodnar.

Summary: A review of the relationship between dopamine and learned flavor preferences in light of the current obesity epidemic.

Question: What role does dopamine (DA) play in conditioned flavor preferences?

Past questions:

• Introduction
• Study 1
• Question(s):
• Does the sweet taste of a noncaloric saccharin solution stimulate DA release?
• Is this response altered in animals that developed a conditioned aversion to saccharin?
• Result(s):
• Intraoral infusions of saccharin stimulated nucleus accumbens (NAc) DA in naive rats, but the  same saccharin infusions significantly reduced DA release in rats previously conditioned to avoid the sweet solution by pairing it with lithium chloride (LiCl) injections.
• Conclusion(s):
• The NAc DA response is/was related to the positive reward quality of the saccharin solution, not to its sweet taste per se, or alternatively, its arousing properties.
• Study 2
• Question(s):
• Does a conditioned taste preference increase NAc DA release, i.e., would it have the opposite effect of a conditioned taste aversion.
• Method(s):
• Two Groups
• 20 hours/day training sessions
• Experimental Pairings
• Received IG Polycose infusions along with a bitter solution
• Received H₂O infusions along with a sour solution (citric acid)
• Control
• Received bitter and sour solutions sans IG infusions
• Result(s):
• Conclusion(s):
• Systemic Studies
• Central Studies



• Caption for figure 1 here...

The Tar Files

Some construction workers decided it would a good idea to boil up some tar over an open flame, right next to the propane tank fueling said flame.

In other news, I've decided it would be a good idea to learn GNU Emacs.

Hello, World!

This is my first blog post. Testing out the blog...