Search

Predicting positive or negative reinforcement from environmental clues is essential to guide decision making and goal-directed behavior. In insect brains the mushroom body is a central structure for learning such valuable associations between sensory signals and reinforcement. We propose a biologically realistic spiking network model of theDrosophilalarval olfactory pathway for the association of odors and reinforcement to bias behavior towards either approach or avoidance. We demonstrate that prediction error coding through integration of present and expected reinforcement in dopaminergic neurons can serve as a driving force in learning that can, combined with synaptic homeostasis, account for the experimentally observed features of acquisition and extinction of associations that depend on the intensity of odor and reward, as well as temporal features of the odor/reward pairing. To allow for a direct comparison of our simulation results with behavioral data we model learning-induced plasticity over the full time course of behavioral experiments and simulate locomotion of individual larvae towards or away from odor sources in a virtual environment.

Larvae of the fruit flyDrosophila melanogasterare a powerful study case for understanding the neural circuits underlying behavior. Indeed, the numerical simplicity of the larval brain has permitted the reconstruction of its synaptic connectome, and genetic tools for manipulating single, identified neurons allow neural circuit function to be investigated with relative ease and precision. We focus on one of the most complex neurons in the brain of the larva (of either sex), the GABAergic anterior paired lateral neuron (APL). Using behavioral and connectomic analyses, optogenetics, Ca2+imaging and pharmacology, we study how APL affects associative olfactory memory. We first provide a detailed account of the structure, regional polarity, connectivity, and metamorphic development of APL, and further confirm that optogenetic activation of APL has an inhibiting effect on its main targets, the mushroom body Kenyon cells. All these findings are consistent with the previously identified function of APL in the sparsening of sensory representations. To our surprise, however, we found that optogenetically activating APL can also have a strong rewarding effect. Specifically, APL activation together with odor presentation establishes an odor-specific, appetitive, associative short-term memory, whereas naïve olfactory behavior remains unaffected. An acute, systemic inhibition of dopamine synthesis as well as an ablation of the dopaminergic pPAM neurons impair reward learning through APL activation. Our findings provide a study case of complex circuit function in a numerically simple brain, and suggest a previously unrecognized capacity of central-brain GABAergic neurons to engage in dopaminergic reinforcement.Significance statementThe single, identified giant anterior paired lateral (APL) neuron is one of the most complex neurons in the insect brain. It is GABAergic and contributes to the sparsening of neuronal activity in the mushroom body, the memory center of insects. We provide the most detailed account yet of the structure of APL in larvalDrosophilaas a neurogenetically accessible study case. We further reveal that, contrary to expectations, the experimental activation of APL can exert a rewarding effect, likely via dopaminergic reward pathways. The present study both provides an example of unexpected circuit complexity in a numerically simple brain, and reports an unexpected effect of activity in central-brain GABAergic circuits.

Neuronally orchestrated muscular movement and locomotion are defining faculties of multicellular animals. Due to its numerically simple brain and neuromuscular system and its genetic accessibility, the larva of the fruit fly Drosophila melanogaster is an established model to study these processes at tractable levels of complexity. However, although the faculty of locomotion clearly pertains to the individual animal, present studies of locomotion in larval Drosophila mostly use group assays and measurements aggregated across individual animals. The alternative is to measure animals one at a time, an extravagance for larger-scale analyses. In principle or in practice, this in particular rules out grasping the inter- and intra-individual variability in locomotion and its genetic and neuronal determinants. Here we present the IMBA (Individual Maggot Behaviour Analyser) for tracking and analysing the behaviour of individual larvae within groups. Using a combination of computational modelling and statistical approaches, the IMBA reliably resolves individual identity across collisions. It does not require specific hardware and can therefore be used in non-expert labs. We take advantage of the IMBA first to systematically describe the inter- and intra-individual variability in free, unconstrained locomotion in wild-type animals. We then report the discovery of a novel, complex locomotion phenotype of a mutant lacking an adhesion-type GPCR. The IMBA further allows us to determine, at the level of individual animals, the modulation of locomotion across repeated activations of dopamine neurons. Strikingly, IMBA can also be used to analyse ‘silly walks’, that is patterns of locomotion it was not originally designed to investigate. This is shown for the transient backward locomotion induced by brief optogenetic activation of the brain-descending ‘mooncrawler’ neurons, and the variability in this behaviour. Thus, the IMBA is an easy-to-use toolbox allowing an unprecedentedly rich view of the behaviour and behavioural variability of individual Drosophila larvae, with utility in multiple biomedical research contexts.

Humans and animals alike form oppositely valenced memories for stimuli that predict the occurrence versus the termination of a reward: appetitive ‘reward’ memory for stimuli associated with the occurrence of a reward and aversive ‘frustration’ memory for stimuli that are associated with its termination. We characterize these memories in larval Drosophila using a combination of Pavlovian conditioning, optogenetic activation of the dopaminergic central-brain DAN-i1864 neuron, and high-resolution video-tracking. This reveals their dependency on the number of training trials and the duration of DAN-i1864 activation, their temporal stability, and the parameters of locomotion that are modulated during memory expression. Together with previous results on ‘punishment’ versus ‘relief’ learning by DAN-f1 neuron activation, this reveals a 2×2 matrix of timing-dependent memory valence for the occurrence/ termination of reward/ punishment. These findings should aid the understanding and modelling of how brains decipher the predictive, causal structure of events around a target reinforcing occurrence.

Animals of many species are capable of “small data” learning, that is, of learning without repetition. Here we introduce larval Drosophila melanogaster as a relatively simple study case for such one-trial learning. Using odor-food associative conditioning, we first show that a sugar that is both sweet and nutritious (fructose) and sugars that are only sweet (arabinose) or only nutritious (sorbitol) all support appetitive one-trial learning. The same is the case for the optogenetic activation of a subset of dopaminergic neurons innervating the mushroom body, the memory center of the insects. In contrast, no one-trial learning is observed for an amino acid reward (aspartic acid). As regards the aversive domain, one-trial learning is demonstrated for high-concentration sodium chloride, but is not observed for a bitter tastant (quinine). Second, we provide follow-up, parametric analyses of odor-fructose learning. Specifically, we ascertain its dependency on the number and duration of training trials, the requirements for the behavioral expression of one-trial odor-fructose memory, its temporal stability, and the feasibility of one-trial differential conditioning. Our results set the stage for a neurogenetic analysis of one-trial learning and define the requirements for modeling mnemonic processes in the larva.

Modulatory (e.g. dopaminergic) neurons provide “teaching signals” that drive associative learning across the animal kingdom, but the circuits that regulate their activity and compute teaching signals are still poorly understood. We provide the first synaptic-resolution connectome of the circuitry upstream of all modulatory neurons in a brain center for associative learning, the mushroom body (MB) of theDrosophilalarva. We discovered afferent pathways from sensory neurons, as well as an unexpected large population of 61 feedback neuron pairs that provide one- and two-step feedback from MB output neurons. The majority of these feedback pathways link distinct memory systems (e.g. aversive and appetitive). We functionally confirmed some of the structural pathways and found that some modulatory neurons compare inhibitory input from their own compartment and excitatory input from compartments of opposite valence, enabling them to compute integrated common-currency predicted values across aversive and appetitive memory systems. This architecture suggests that the MB functions as an interconnected ensemble during learning and that distinct types of previously formed memories can regulate future learning about a stimulus. We developed a model of the circuit constrained by the connectome and by the functional data which revealed that the newly discovered architectural motifs, namely the multilevel feedback architecture and the extensive cross-compartment connections, increase the computational performance and flexibility on learning tasks. Together our study provides the most detailed view to date of a recurrent brain circuit that computes teaching signals and provides insights into the architectural motifs that support reinforcement learning in a biological system.

Two things are worth remembering about an aversive event: What made it happen? What made it cease? If a stimulus precedes an aversive event, it becomes a signal for threat and will later elicit behavior indicating conditioned fear. However, if the stimulus is presented upon cessation of the aversive event, it elicits behavior indicating conditioned “relief.” What are the neuronal bases for such learning? Using functional magnetic resonance imaging (fMRI) in humans we found that a fear-conditioned stimulus activates amygdala but not striatum, whereas a relief-conditioned stimulus activates striatum but not amygdala. Correspondingly, acute inactivation of amygdala or of ventral striatum in rats respectively abolished only conditioned fear or only conditioned relief. Thus, the behaviorally opponent memories supported by onset and offset of aversive events engage and require fear and reward networks, respectively. This may explain attraction to stimuli associated with the cessation of trauma or of panic attacks.

Synapsin is an evolutionarily conserved, presynaptic vesicular phosphoprotein. Here, we ask where and how synapsin functions in associative behavioral plasticity. Upon loss or reduction of synapsin in a deletion mutant or via RNAi, respectively, Drosophila larvae are impaired in odor-sugar associative learning. Acute global expression of synapsin and local expression in only the mushroom body, a third-order “cortical” brain region, fully restores associative ability in the mutant. No rescue is found by synapsin expression in mushroom body input neurons or by expression excluding the mushroom bodies. On the molecular level, we find that a transgenically expressed synapsin with dysfunctional PKA-consensus sites cannot rescue the defect of the mutant in associative function, thus assigning synapsin as a behaviorally relevant effector of the AC-cAMP-PKA cascade. We therefore suggest that synapsin acts in associative memory trace formation in the mushroom bodies, as a downstream element of AC-cAMP-PKA signaling. These analyses provide a comprehensive chain of explanation from the molecular level to an associative behavioral change.

Odor-shock memory in Drosophila melanogaster consists of heterogeneous components each with different dynamics. We report that a null mutant for the evolutionarily conserved synaptic protein Synapsin entails a memory deficit selectively in early memory, leaving later memory as well as sensory motor function unaffected. Notably, a consolidated memory component remaining after cold-anesthesia is not impaired, suggesting that only anesthesia-sensitive memory [ASM] depends on Synapsin. The lack of Synapsin does not further impair the memory deficit of mutants for the rutabaga gene encoding the type I adenylyl cyclase. This suggests that cAMP signaling, through a Synapsin-dependent mechanism, may underlie the formation of a labile memory component.

Blocking occurs when previous training with a stimulus A reduces (blocks) subsequent learning about a stimulus B, when A and B are trained in compound. The question of whether blocking exists in olfactory conditioning of proboscis extension reflex (PER) in honeybees is under debate. The last published accounts on blocking in honeybees state that blocking occurs when odors A and B are similar (the “similarity hypothesis”). We have tested this hypothesis using four odors (1-octanol, 1-nonanol, eugenol, and limonene) chosen on the basis of their chemical and physiological similarity (experiment 1). We established a generalization matrix that measured perceptual similarity. Bees in the “block group” were first trained with an odor A and, in the second phase, with the mixture AB. Bees in the “novel group” (control group) were first trained with an odor N and, in the second phase, with the mixture AB. After conditioning, bees in both groups were tested for their response to B. We assayed all 24 possible combinations for the four odors standing for A, B, and N. We found blocking in four cases, augmentation in two cases, and no difference in 18 cases; odor similarity could not account for these results. We also repeated the experiments with those six odor combinations that gave rise to the similarity hypothesis (experiment 2: 1-hexanol, 1-octanol, geraniol) and found augmentation in one and no effect in five cases. Thus, blocking is not a consistent phenomenon, nor does it depend on odor similarity.