Search

Punishment feels bad, but relief upon its termination feels good. As a consequence of such timing-dependent valence reversal, memories of opposite valence can result from associating stimulus A with, for example, the occurrence of punishment (A−) versus punishment termination (−A): A− training results in aversive memory, but −A training in appetitive memory (corresponding effects exist for reward occurrence and termination). Whereas learning through the occurrence of punishment is well studied, much less is known about learning through its termination. Current research investigates how dopaminergic system function contributes to these processes in Drosophila, rats and humans. We argue that dopamine-related psychopathology may entail distortions in learning through punishment termination, and that this may contribute, for example, to non-suicidal self-injury or post-traumatic stress disorder.

Painful events establish opponent memories: cues that precede pain are remembered negatively, whereas cues that follow pain, thus coinciding with relief are recalled positively. How do individual reinforcement-signaling neurons contribute to this “timing-dependent valence-reversal?” We addressed this question using an optogenetic approach in the fruit fly. Two types of fly dopaminergic neuron, each comprising just one paired cell, indeed established learned avoidance of odors that preceded their photostimulation during training, and learned approach to odors that followed the photostimulation. This is in striking parallel to punishment versus relief memories reinforced by a real noxious event. For only one of these neuron types, both effects were strong enough for further analyses. Notably, interfering with dopamine biosynthesis in these neurons partially impaired the punishing effect, but not the relieving after-effect of their photostimulation. We discuss how this finding constraints existing computational models of punishment versus relief memories and introduce a new model, which also incorporates findings from mammals. Furthermore, whether using dopaminergic neuron photostimulation or a real noxious event, more prolonged punishment led to stronger relief. This parametric feature of relief may also apply to other animals and may explain particular aspects of related behavioral dysfunction in humans.

Mapping brain function to brain structure is a fundamental task for neuroscience. For such an endeavour, the Drosophila larva is simple enough to be tractable, yet complex enough to be interesting. It features about 10,000 neurons and is capable of various taxes, kineses and Pavlovian conditioning. All its neurons are currently being mapped into a light-microscopical atlas, and Gal4 strains are being generated to experimentally access neurons one at a time. In addition, an electron microscopic reconstruction of its nervous system seems within reach. Notably, this electron microscope-based connectome is being drafted for a stage 1 larva - because stage 1 larvae are much smaller than stage 3 larvae. However, most behaviour analyses have been performed for stage 3 larvae because their larger size makes them easier to handle and observe. It is therefore warranted to either redo the electron microscopic reconstruction for a stage 3 larva or to survey the behavioural faculties of stage 1 larvae. We provide the latter. In a community-based approach we called the Ol1mpiad, we probed stage 1 Drosophila larvae for free locomotion, feeding, responsiveness to substrate vibration, gentle and nociceptive touch, burrowing, olfactory preference and thermotaxis, light avoidance, gustatory choice of various tastants plus odour-taste associative learning, as well as light/dark-electric shock associative learning. Quantitatively, stage 1 larvae show lower scores in most tasks, arguably because of their smaller size and lower speed. Qualitatively, however, stage 1 larvae perform strikingly similar to stage 3 larvae in almost all cases. These results bolster confidence in mapping brain structure and behaviour across developmental stages. The Gulbenkian/Melbourne group was supported by the Fundação para a Ciência e a Tecnologia (FCT; post-doctoral fellowship to M.J.A.-C.: SFRH/BPD/75993/2011; exploratory grant to M.J.A.-C. and C.K.M.: EXPL/BEX-BID/0497/2013). The Münster group was supported by the Cluster of Excellence Cells in Motion (CiM) and the CiM International Max Planck research school (CiM-IMPRS). The Barcelona group was supported by the Spanish Ministry of Economy and Competitiveness, 'Centro de Excelencia Severo Ochoa 2013-2017' (SEV-2012-0208), the CERCA Programme/Generalitat de Catalunya, the 'la Caixa' International PhD Programme, and the Spanish Ministry of Science and Innovation (BFU2011-26208). The LIN-GoLM group received support from the Wissenschaftsgemeinschaft Gottfried Wilhelm Leibniz, the State of Sachsen-Anhalt, the Center for Behavioral Brain Sciences Magdeburg and the Otto von Guericke Universität Magdeburg, as well as from the Deutsche Forschungsgemeinschaft (CRC 779 Motivated behaviour: B11; GE1091/4-1) and the European Commission (FP7-ICT project Miniature Insect Model for Active Learning MINIMAL). The Janelia group received support from the Howard Hughes Medical Institute. The Fribourg group was supported by a starter grant from the European Research Council (ERC-2012-StG 309832-PhotoNaviNet) and the Swiss National Science Foundation (31003A_169993). The Hamburg group was supported by the Deutsche Forschungsgemeinschaft (SPP 1926, Next generation optogenetics, SO1337/2-1), and the Landesforschungsförderung Hamburg (LFF-FV27). The LIN-MolSysBiol group received support from the Wissenschaftsgemeinschaft Gottfried Wilhelm Leibniz, the State of Sachsen-Anhalt, the Center for Behavioral Brain Sciences Magdeburg and the Deutsche Forschungsgemeinschaft (CRC 779 Motivated behaviour: B15; YA272/2-1). The Bonn group was supported by the Deutsche Forschungsgemeinschaft (PA 787/7-1) and Cluster of Excellence ImmunoSensation. The Konstanz group was supported by Deutsche Forschungsgemeinschaft (TH1584/1-1, TH1584/3-1), the Baden-Württemberg Stiftung and the Zukunftskolleg of the University of Konstanz.

Larval Drosophila offer a study case for behavioral neurogenetics that is simple enough to be experimentally tractable, yet complex enough to be worth the effort. We provide a detailed, hands-on manual for Pavlovian odor-reward learning in these animals. Given the versatility of Drosophila for genetic analyses, combined with the evolutionarily shared genetic heritage with humans, the paradigm has utility not only in behavioral neurogenetics and experimental psychology, but for translational biomedicine as well. Together with the upcoming total synaptic connectome of the Drosophila nervous system and the possibilities of single-cell-specific transgene expression, it offers enticing opportunities for research. Indeed, the paradigm has already been adopted by a number of labs and is robust enough to be used for teaching in classroom settings. This has given rise to a demand for a detailed, hands-on manual directed at newcomers and/or at laboratory novices, and this is what we here provide. The paradigm and the present manual have a unique set of features: (1) The paradigm is cheap, easy, and robust; (2) The manual is detailed enough for newcomers or laboratory novices; (3) It briefly covers the essential scientific context; (4) It includes sheets for scoring, data analysis, and display; (5) It is multilingual: in addition to an English version we provide German, French, Japanese, Spanish and Italian language versions as well. The present manual can thus foster science education at an earlier age and enable research by a broader community than has been the case to date.

New techniques and resources for remote control of neuronal activity at single-cell resolution have led to an explosion of research into the behavioral neurogenetics of the memory trace (engram) in the fruit fly Drosophila . The available data suggest that after odor-punishment training, associative and odor-specific engrams are local to a single layer of processing along the olfactory pathway, the presynaptic output terminals of the mushroom body Kenyon cells. At the same time, engrams are distributed across this layer for different mnemonic aspects of such a training event, and for different kinds of training event. This nature of engrams as being local and distributed can be revealed at due resolution in Drosophila – as can the way learned behavior is organized in an outcome-oriented way.

Two things are worth remembering about an aversive event: What made it happen? What made it cease? If a stimulus precedes an aversive event, it becomes a signal for threat and will later elicit behavior indicating conditioned fear. However, if the stimulus is presented upon cessation of the aversive event, it elicits behavior indicating conditioned “relief.” What are the neuronal bases for such learning? Using functional magnetic resonance imaging (fMRI) in humans we found that a fear-conditioned stimulus activates amygdala but not striatum, whereas a relief-conditioned stimulus activates striatum but not amygdala. Correspondingly, acute inactivation of amygdala or of ventral striatum in rats respectively abolished only conditioned fear or only conditioned relief. Thus, the behaviorally opponent memories supported by onset and offset of aversive events engage and require fear and reward networks, respectively. This may explain attraction to stimuli associated with the cessation of trauma or of panic attacks.

Pain is aversive, but does the cessation of pain (‘relief’) have a reward-like effect? Indeed, fruitflies avoid an odour previously presented before a painful event, but approach an odour previously presented after a painful event. Thus, event-timing may turn punishment to reward. However, is event-timing also crucial in humans who can have explicit cognitions about associations? Here, we show that stimuli associated with pain-relief acquire positive implicit valence but are explicitly rated as aversive. Specifically, the startle response, an evolutionarily conserved defence reflex, is attenuated by stimuli that had previously followed a painful event, indicating implicit positive valence of the conditioned stimulus; nevertheless, participants explicitly evaluate these stimuli as ‘emotionally negative’. These results demonstrate a rift between the implicit and explicit conditioned valence induced by pain relief. They might explain why humans in some cases are attracted by conditioned stimuli despite explicitly judging them as negative.

Odor-shock memory in Drosophila melanogaster consists of heterogeneous components each with different dynamics. We report that a null mutant for the evolutionarily conserved synaptic protein Synapsin entails a memory deficit selectively in early memory, leaving later memory as well as sensory motor function unaffected. Notably, a consolidated memory component remaining after cold-anesthesia is not impaired, suggesting that only anesthesia-sensitive memory [ASM] depends on Synapsin. The lack of Synapsin does not further impair the memory deficit of mutants for the rutabaga gene encoding the type I adenylyl cyclase. This suggests that cAMP signaling, through a Synapsin-dependent mechanism, may underlie the formation of a labile memory component.

An experience with electric shock can support two opposing kinds of behavioral effects: Stimuli that precede shock during training are subsequently avoided as predictors for punishment, whereas stimuli that follow shock during training are later on approached, as they predict relief. We show here, for the fruit fly Drosophila, that upon the loss of white-function, the balance between these two kinds of learning is distorted in favor of punishment learning: white1118 mutants show stronger punishment learning and weaker relief learning, as compared to wild type flies. Thus, white1118 mutants establish, overall, more "negative" memories for the shock experience. This only concerns the mnemonic effects of the shock; the immediate, reflexive responsiveness to shock remains unaltered. Also, learning about reward is apparently unaffected, both in adult and larval Drosophila. Prompted by the proposed function of the White protein as the transporter for biogenic amine precursors, we probed the brains of white1118 mutants for the amounts of biogenic amines (octopamine, tyramine, dopamine, and serotonin) by using high-pressure liquid chromatography coupled to mass spectrometry. Using this method, we found, however, no difference between white1118 and wild type files for any of the probed amines. In any event, analyses of how the white1118 mutation affects the balance between punishment and relief learning should provide a study case of how heritable distortions of such balance can come about. Finally, the effects of the white1118 mutation should be considered as a source of confound when using white as the "marker gene" in behavior-genetic analyses of any sort.

We studied the behavioural consequences of ‘traumatic’, painful experiences. These consequences were fundamentally asymmetric. Fruit flies, Drosophila melanogaster , learned two kinds of prediction regarding a ‘traumatic’ experience. If an odour preceded an electric shock during training, it predicted shock, and flies subsequently avoided it. When the sequence of events during training was reversed, that is odour followed shock, the odour predicted relief from shock and flies approached it. We call this latter effect ‘relief’ learning and showed that, in terms of psychological mechanisms, it established genuinely associative conditioned approach behaviour. Parametric analyses showed that relief learning was reproducible across experimenters; it did not depend on the flies' gender and reached asymptotic levels after six training trials. Of five chosen odour-pairs, two supported relief learning at all concentrations tested; for one odour-pair, we observed optimal relief learning at an intermediate odour concentration; for two odour-pairs, relief learning could not be demonstrated. Furthermore, relief learning was maximal with relatively mild shocks, supporting stable retention for the first 2 h after training. Knowledge of these parametric features should aid uncovering relief learning in other experimental systems. In terms of psychological mechanism, context–shock pretraining had no effect on subsequent relief learning, suggesting that it is not mediated by context associations. These analyses may further our understanding of the psychological mechanisms underlying behavioural changes after traumatic experience. They facilitate research into the neurobiology of pain relief learning, enabling the implementation of truly bioinspired learning rules for technical devices.

How is behaviour organised across sensory modalities? Specifically, we ask concerning the fruit fly Drosophila melanogaster how visual context affects olfactory learning and recall and whether information about visual context is getting integrated into olfactory memory. We find that changing visual context between training and test does not deteriorate olfactory memory scores, suggesting that these olfactory memories can drive behaviour despite a mismatch of visual context between training and test. Rather, both the establishment and the recall of olfactory memory are generally facilitated by light. In a follow-up experiment, we find no evidence for learning about combinations of odours and visual context as predictors for reinforcement even after explicit training in a so-called biconditional discrimination task. Thus, a ‘true’ interaction between visual and olfactory modalities is not evident; instead, light seems to influence olfactory learning and recall unspecifically, for example by altering motor activity, alertness or olfactory acuity.

How do animals adaptively integrate innate with learned behavioral tendencies? We tackle this question using chemotaxis as a paradigm. Chemotaxis in the Drosophila larva largely results from a sequence of runs and oriented turns. Thus, the larvae minimally need to determine (i) how fast to run, (ii) when to initiate a turn, and (iii) where to direct a turn. We first report how odor-source intensities modulate these decisions to bring about higher levels of chemotactic performance for higher odor-source intensities during innate chemotaxis. We then examine whether the same modulations are responsible for alterations of chemotactic performance by learned odor "valence" (understood throughout as level of attractiveness). We find that run speed (i) is neither modulated by the innate nor by the learned valence of an odor. Turn rate (ii), however, is modulated by both: the higher the innate or learned valence of the odor, the less often larvae turn whenever heading toward the odor source, and the more often they turn when heading away. Likewise, turning direction (iii) is modulated concordantly by innate and learned valence: turning is biased more strongly toward the odor source when either innate or learned valence is high. Using numerical simulations, we show that a modulation of both turn rate and of turning direction is sufficient to account for the empirically found differences in preference scores across experimental conditions. Our results suggest that innate and learned valence organize adaptive olfactory search behavior by their summed effects on turn rate and turning direction, but not on run speed. This work should aid studies into the neural mechanisms by which memory impacts specific aspects of behavior. We received project support by the European Commission (FP7-ICT project Miniature Insect Model for Active Learning [MINIMAL]). M.L. and S.F.R. acknowledge further funding from the Spanish Ministry of Science and Innovation (MICINN, BFU2011-483 26208) and the EMBL/CRG Systems Biology Program. A.D. acknowledges support via the grants EP/F500385/1 and BB/F529254/1 for the University of Edinburgh School of Informatics Doctoral Training Centre in Neuroinformatics and Computational Neuroscience, the UK Engineering and Physical Sciences Research Council (EPSRC), UK Biotechnology and Biological Sciences Research Council (BBSRC), and the UK Medical Research Council (MRC). B.G. received additional support from the Deutsche Forschungsgemeinschaft (DFG) (SFB 779 Motivated behavior), and the Bundesministerium fu¨r Bildung und Forschung (BMBF) (Bernstein Network Insect inspired robotics).

Associative plasticity is a basic characteristic of behaviour. We analysed associative olfactory learning in larval Drosophila, using a reciprocal conditioning assay. One group of animals was rewarded with fructose in the presence of odour A but not in the presence of odour B (A+/B); the companion group received reciprocal training (A/B+). During the test, larvae were given a choice between A and B; those that had received A+/B training showed a higher preference for A than reciprocally trained larvae did. As all other parameters were equal between groups, this difference was exclusively due to associative learning. Learning reached an asymptote after as few as three training trials and 2.0 mol/litre of fructose yielded asymptotic learning. Learning was not modulated by larval age tested 4, 5 and 6 days after egg laying or by gender. En masse assays confirmed the lack of gender differences. Memory was fully stable for at least 30 min. These experiments provide a basis for future investigations into the cellular and molecular underpinnings of appetitive olfactory learning in larval Drosophila.

Blocking occurs when previous training with a stimulus A reduces (blocks) subsequent learning about a stimulus B, when A and B are trained in compound. The question of whether blocking exists in olfactory conditioning of proboscis extension reflex (PER) in honeybees is under debate. The last published accounts on blocking in honeybees state that blocking occurs when odors A and B are similar (the “similarity hypothesis”). We have tested this hypothesis using four odors (1-octanol, 1-nonanol, eugenol, and limonene) chosen on the basis of their chemical and physiological similarity (experiment 1). We established a generalization matrix that measured perceptual similarity. Bees in the “block group” were first trained with an odor A and, in the second phase, with the mixture AB. Bees in the “novel group” (control group) were first trained with an odor N and, in the second phase, with the mixture AB. After conditioning, bees in both groups were tested for their response to B. We assayed all 24 possible combinations for the four odors standing for A, B, and N. We found blocking in four cases, augmentation in two cases, and no difference in 18 cases; odor similarity could not account for these results. We also repeated the experiments with those six odor combinations that gave rise to the similarity hypothesis (experiment 2: 1-hexanol, 1-octanol, geraniol) and found augmentation in one and no effect in five cases. Thus, blocking is not a consistent phenomenon, nor does it depend on odor similarity.

The only property of reinforcement insects are commonly thought to learn about is its value. We show that larval Drosophila not only remember the value of reinforcement (How much?), but also its quality (What?). This is demonstrated both within the appetitive domain by using sugar vs amino acid as different reward qualities, and within the aversive domain by using bitter vs high-concentration salt as different qualities of punishment. From the available literature, such nuanced memories for the quality of reinforcement are unexpected and pose a challenge to present models of how insect memory is organized. Given that animals as simple as larval Drosophila, endowed with but 10,000 neurons, operate with both reinforcement value and quality, we suggest that both are fundamental aspects of mnemonic processing—in any brain. DOI: http://dx.doi.org/10.7554/eLife.04711.001, eLife digest Actions have consequences; positive consequences or rewards make it more likely that a behavior will be repeated, while negative consequences or punishments can stop a behavior occurring again. Neuroscientists commonly refer to such rewards and punishments as ‘reinforcement’. Fruit flies that are given a reward of sugar when they experience an odor will move towards the odor in later tests. However, in 2011, research revealed that if the flies were given at least the same amount of sugar in the tests as they were rewarded with during the earlier training, the flies stopped moving towards the odor. This suggests that fruit flies can recall how strong a reward was in the past and compare this remembered strength to the current reward on offer; fruit flies will only continue searching if they expect to gain a larger reward by doing so. Insects were commonly thought to only learn the amount or ‘value’ of reinforcement, but not recall what kind or ‘quality’ of reward (or punishment) they had experienced. Now Schleyer et al.—including some of the researchers involved in the 2011 work—challenge and extend this notion and show that fruit fly larvae can remember both the value and quality of rewards and punishments. Fruit fly larvae were trained to expect a reward of sugar when exposed to one odor and nothing when exposed to a different odor. Consistent with the previous results, the larvae moved towards the first odor in the tests where no additional reward was provided. Moreover, the larvae did not move towards the odor in later tests if an equal or greater amount of sugar was provided during the testing stage. Schleyer et al. then took larvae that had been trained to expect a sugar reward and gave them a different, but equally valuable, reward during the testing stage—in this case, the reward was an amino acid called aspartic acid. These experiments revealed that most of the larvae continued to move towards the sugar-associated odor in search of the sugar reward. This indicates that the larvae were able to remember the quality of the reward, namely that it was sugar rather than aspartic acid. Schleyer et al. performed similar experiments, and observed similar results, when using two different punishments: bitter-tasting quinine and high concentrations of salt. These findings show that experiencing an odor along with taste reinforcement could set up a memory specific to the quality of reinforcement in fruit fly larvae. Given the numerical simplicity of a larva's brain—which contains only 10,000 neurons—it is likely that other animals can also recall both the value and quality of a reward or punishment. However, understanding how such specificity comes about should be easier in the larva's simple brain. DOI: http://dx.doi.org/10.7554/eLife.04711.002

Adverse life events can induce two kinds of memory with opposite valence, dependent on timing: “negative” memories for stimuli preceding them and “positive” memories for stimuli experienced at the moment of “relief.” Such punishment memory and relief memory are found in insects, rats, and man. For example, fruit flies (Drosophila melanogaster) avoid an odor after odor-shock training (“forward conditioning” of the odor), whereas after shock-odor training (“backward conditioning” of the odor) they approach it. Do these timing-dependent associative processes share molecular determinants? We focus on the role of Synapsin, a conserved presynaptic phosphoprotein regulating the balance between the reserve pool and the readily releasable pool of synaptic vesicles. We find that a lack of Synapsin leaves task-relevant sensory and motor faculties unaffected. In contrast, both punishment memory and relief memory scores are reduced. These defects reflect a true lessening of associative memory strength, as distortions in nonassociative processing (e.g., susceptibility to handling, adaptation, habituation, sensitization), discrimination ability, and changes in the time course of coincidence detection can be ruled out as alternative explanations. Reductions in punishment- and relief-memory strength are also observed upon an RNAi-mediated knock-down of Synapsin, and are rescued both by acutely restoring Synapsin and by locally restoring it in the mushroom bodies of mutant flies. Thus, both punishment memory and relief memory require the Synapsin protein and in this sense share genetic and molecular determinants. We note that corresponding molecular commonalities between punishment memory and relief memory in humans would constrain pharmacological attempts to selectively interfere with excessive associative punishment memories, e.g., after traumatic experiences.

Similarity between odours is notoriously difficult to measure. Widely used behavioural approaches in insect olfaction research are cross-adaptation, masking, as well as associative tasks based on olfactory learning and the subsequent testing for how specific the established memory is. A concern with such memory-based approaches is that the learning process required to establish an odour memory may alter the way the odour is processed, such that measures of perception taken at the test are distorted. The present study was therefore designed to see whether behavioural judgements of perceptual distance are different for two different memory-based tasks, namely generalization and discrimination. We used odour–reward learning in larval Drosophila as a study case. In order to challenge the larvae's olfactory system, we chose to work with binary mixtures and their elements (1-octanol, n-amyl acetate, 3-octanol, benzaldehyde and hexyl acetate). We determined the perceptual distance between each mixture and its elements, first in a generalization task, and then in a discrimination task. It turns out that scores of perceptual distance are correlated between both tasks. A re-analysis of published studies looking at element-to-element perceptual distances in larval reward learning and in adult punishment learning confirms this result. We therefore suggest that across a given set of olfactory stimuli, associative training does not grossly alter the pattern of perceptual distances.

Memories relating to a painful, negative event are adaptive and can be stored for a lifetime to support preemptive avoidance, escape, or attack behavior. However, under unfavorable circumstances such memories can become overwhelmingly powerful. They may trigger excessively negative psychological states and uncontrollable avoidance of locations, objects, or social interactions. It is therefore obvious that any process to counteract such effects will be of value. In this context, we stress from a basic-research perspective that painful, negative events are “Janus-faced” in the sense that there are actually two aspects about them that are worth remembering: What made them happen and what made them cease. We review published findings from fruit flies, rats, and man showing that both aspects, respectively related to the onset and the offset of the negative event, induce distinct and oppositely valenced memories: Stimuli experienced before an electric shock acquire negative valence as they signal upcoming punishment, whereas stimuli experienced after an electric shock acquire positive valence because of their association with the relieving cessation of pain. We discuss how memories for such punishment- and relief-learning are organized, how this organization fits into the threat-imminence model of defensive behavior, and what perspectives these considerations offer for applied psychology in the context of trauma, panic, and nonsuicidal self-injury.

Reserpine depletes biogenic amines from their stores in the honeybee (Apis mellifera carnica) brain and leads to impaired appetitive conditioning using sucrose as a reinforcer. Compensatory injection of octopamine or dopamine directly into the brain restores these behavioral losses. Dopamine rescues the slowing-down effect on motor patterns, but not sensitization or conditioning. Octopamine leaves the motor patterns as well as sensitization unchanged but rescues conditioning. Specifically, octopamine rescues acquisition but not retrieval. Serotonin has no significant effect on sensitization but impairs conditioning. The authors conclude that octopamine is involved in selectively mediating the reinforcing but not the sensitizing or response-releasing function of the sucrose reward, whereas dopamine is selectively involved in the expression of the motor response.

Although work in a wide variety of species and paradigms has demonstrated that long-term memory is sensitive to the blocking of protein synthesis, previous studies have suggested that the honeybee might represent an exception to this rule. Retention tested one day after training was not impaired by the inhibition of translation by cycloheximide. Using blockers of either transcription (actinomycin D) or translation (anisomycin), we present experiments that reconcile this unusual finding by testing over longer retention periods. Honeybees were conditioned to associate an odourant with a sucrose reward. Typically, this leads to stable retention over days. However, injection of either drug led to lower retention after 4 days, whereas retention after 2 or sometimes even 3 days was unaffected. This dissociates two forms of memory: a protein synthesis-independent, medium-term memory (up to 3 days) and a protein synthesis-dependent, long-term memory lasting for at least 4 days.

We use classical conditioning of the honeybee (Apis mellifera) proboscis extension reflex with a visual (A) and an olfactory (X) conditioned stimulus in a blocking paradigm. Typically, learning about one element (X) of a compound (AX) is decreased (blocked) if the other component (A) has previously been rewarded alone. Our results show that visual pretraining did not produce blocking in honeybees: instead, forward pairings of A with a reward increased subsequent learning about X relative to a backward pairing control. This finding violates the independence assumption, which holds that elements of inter-modal compound stimuli change associative strength independently of each other. Furthermore, it is at odds with common theories of conditioning that predict blocking and assume that the elements of a compound stimulus rely on one common internal reinforcing signal. Taking the functional anatomy of the honeybee brain into account, we suggest that vision and olfaction may not rely on the same internal reinforcing signal; compound interactions might thus reflect the wiring of the honeybee nervous system and the biological significance of different sensory modalities during natural behaviour.

Stimulus intensity is an important determinant for perception, learning and behaviour. We studied the effects of odorant concentration on classical conditioning involving odorants and odorant-mechanosensory compounds using the proboscis-extension reflex in the honeybee. Our results show that high concentrations of odorant (a) support better discrimination in a feature-positive task using rewarded odorant-mechanosensory compounds versus unrewarded mechanosensory stimuli, (b) have a stronger capacity to overshadow learning of a simultaneously trained mechanosensory stimulus, and (c) induce better memory consolidation. Furthermore, honeybees were trained discriminatively to two different concentrations of one odorant. Honeybees are not able to solve this task when presented with rewarded low versus unrewarded high concentrations. Taken together, our results suggest that high concentrations of odorant support stronger associations (are more ‘salient’) than low concentrations. Our results, however, do not indicate that honeybees can treat two different concentrations of one odorant as qualitatively different stimuli. These findings fill a gap in what is known about honeybee olfactory learning and are a first step in relating behaviour to recent advances in the physiological analysis of coding for odorant concentration in honeybees.

Summary Drosophila larvae are focused on feeding and have few neurons. Within these bounds, however, there still are behavioural degrees of freedom. This review is devoted to what these elements of flexibility are, and how they come about. Regarding odour–food associative learning, the emerging working hypothesis is that when a mushroom body neuron is activated as a part of an odour-specific set of mushroom body neurons, and coincidently receives a reinforcement signal carried by aminergic neurons, the AC-cAMP-PKA cascade is triggered. One substrate of this cascade is Synapsin, and therefore this review features a general and comparative discussion of Synapsin function. Phosphorylation of Synapsin ensures an alteration of synaptic strength between this mushroom body neuron and its target neuron(s). If the trained odour is encountered again, the pattern of mushroom body neurons coding this odour is activated, such that their modified output now allows conditioned behaviour. However, such an activated memory trace does not automatically cause conditioned behaviour. Rather, in a process that remains off-line from behaviour, the larvae compare the value of the testing situation (based on gustatory input) with the value of the odour-activated memory trace (based on mushroom body output). The circuit towards appetitive conditioned behaviour is closed only if the memory trace suggests that tracking down the learned odour will lead to a place better than the current one. It is this expectation of a positive outcome that is the immediate cause of appetitive conditioned behaviour. Such conditioned search for reward corresponds to a view of aversive conditioned behaviour as conditioned escape from punishment, which is enabled only if there is something to escape from – much in the same way as we only search for things that are not there, and run for the emergency exit only when there is an emergency. One may now ask whether beyond ‘value’ additional information about reinforcement is contained in the memory trace, such as information about the kind and intensity of the reinforcer used. The Drosophila larva may allow us to develop satisfyingly detailed accounts of such mnemonic richness – if it exists.

The brain is the organ of behavior organization. It structures the solution to the problem of what to do. This is complicated because usually we cannot be certain which behavior would be relatively the best. These processes, taking place between the moment when an uncertainty between behavioral options is recognized and the actual expression of behavior, we regard as ‘taking a decision.’ Such decision making needs to integrate (1) sensory input, (2) the current status reflecting evolutionary and individual history, (3) the available behavioral options, and (4) their expected outcomes. We focus on the decision to behaviorally express an associative memory trace—or not. After sketching the architecture of the chemobehavioral system in larval Drosophila, we present a working hypothesis of odor–taste associative memory trace formation and then discuss whether outcome expectations contribute to the organization of conditioned behavior. We argue that indeed conditioned olfactory behavior is organized according to its expected outcome, namely toward finding reward or escaping punishment, respectively. Conditioned olfactory behaviors are thus not responsive in nature but, rather, are actions expressed for the sake of the sought-for reward and the attempted relief. In addition to the organization of such outcome expectations, we discuss parametric features (‘axes’) of behavioral tasks that we believe bear upon the decision character of the underlying process and discuss whether these features can be found, or may reasonably be sought for, in larval Drosophila. It is argued that rather than trying to draw a line between behavioral processes that reflect decisions and those that are not, it is more useful to ask how strong the decision character of a given behavioral faculty is?

We tested whether Drosophila larvae can associate odours with a mechanosensory disturbance as a punishment, using substrate vibration conveyed by a loudspeaker (buzz: ![Graphic][1] ). One odour (A) was presented with the buzz, while another odour (B) was presented without the buzz (A![Graphic][2] /B training). Then, animals were offered the choice between A and B. After reciprocal training (A/B![Graphic][3] ), a second experimental group was tested in the same way. We found that larvae show conditioned escape from the previously punished odour. We further report an increase of associative performance scores with the number of punishments, and an increase according to the number of training cycles. Within the range tested (between 50 and 200 Hz), however, the pitch of the buzz does not apparently impact associative success. Last, but not least, we characterized odour–buzz memories with regard to the conditions under which they are behaviourally expressed – or not. In accordance with what has previously been found for associative learning between odours and bad taste (such as high concentration salt or quinine), we report that conditioned escape after odour–buzz learning is disabled if escape is not warranted, i.e. if no punishment to escape from is present during testing. Together with the already established paradigms for the association of odour and bad taste, the present assay offers the prospect of analysing how a relatively simple brain orchestrates memory and behaviour with regard to different kinds of ‘bad’ events. [1]: /embed/inline-graphic-1.gif [2]: /embed/inline-graphic-2.gif [3]: /embed/inline-graphic-3.gif

How do physico-chemical stimulus features, perception, and physiology relate? Given the multi-layered and parallel architecture of brains, the question specifically is where physiological activity patterns correspond to stimulus features and/ or perception. Perceived distances between six odour pairs are defined behaviourally from four independent odour recognition tasks. We find that, in register with the physico-chemical distances of these odours, perceived distances for 3- octanol and n-amylacetate are consistently smallest in all four tasks, while the other five odour pairs are about equally distinct. Optical imaging in the antennal lobe, using a calcium sensor transgenically expressed in only first-order sensory or only second-order olfactory projection neurons, reveals that 3-octanol and n-amylacetate are distinctly represented in sensory neurons, but appear merged in projection neurons. These results may suggest that within-antennal lobe processing funnels sensory signals into behaviourally meaningful categories, in register with the physico-chemical relatedness of the odours. peerReviewed

We investigate olfactory associative learning in larval Drosophila. A reciprocal training design is used, such that one group of animals receives a reward in the presence of odor X but not in the presence of odor Y (Train: X+ // Y), whereas another group is trained reciprocally (Train: X // Y+). After training, differences in odor preference between these reciprocally trained groups in a choice test (Test: X -- Y) reflect associative learning. The current study, after showing which odor pairs can be used for such learning experiments, 1) introduces a one-odor version of such reciprocal paradigm that allows estimating the learnability of single odors. Regarding this reciprocal one-odor paradigm, we show that 2) paired presentations of an odor with a reward increase odor preference above baseline, whereas unpaired presentations of odor and reward decrease odor preference below baseline; this suggests that odors can become predictive either of reward or of reward absence. Furthermore, we show that 3) innate attractiveness and associative learnability can be dissociated. These data deepen our understanding of odor-reward learning in larval Drosophila on the behavioral level, and thus foster its neurogenetic analysis.

To provide a behavior-based estimate of odor similarity in larval Drosophila, we use 4 recognition-type experiments: 1) We train larvae to associate an odor with food and then test whether they would regard another odor as the same as the trained one. 2) We train larvae to associate an odor with food and test whether they prefer the trained odor against a novel nontrained one. 3) We train larvae differentially to associate one odor with food, but not the other one, and test whether they prefer the rewarded against the nonrewarded odor. 4) In an experiment like (3), we test the larvae after a 30-min break. This yields a combined task-independent estimate of perceived difference between odor pairs. Comparing these perceived differences to published measures of physicochemical difference reveals a weak correlation. A notable exception are 3-octanol and benzaldehyde, which are distinct in published accounts of chemical similarity and in terms of their published sensory representation but nevertheless are consistently regarded as the most similar of the 10 odor pairs employed. It thus appears as if at least some aspects of olfactory perception are "computed" in postreceptor circuits on the basis of sensory signals rather than being immediately given by them.

Learnt predictive behavior faces a dilemma: predictive stimuli will never 'replay' exactly as during the learning event, requiring generalization. In turn, minute differences can become meaningful, prompting discrimination. To provide a study case for an adaptive adjustment of this generalization-discrimination balance, the authors ask whether Drosophila melanogaster larvae are able to either generalize or discriminate between two odors (1-octen-3-ol and 3-octanol), depending on the task. The authors find that after discriminatively rewarding one but not the other odor, larvae show conditioned preference for the rewarded odor. On the other hand, no odor specificity is observed after nondiscriminative training, even if the test involves a choice between both odors. Thus, for this odor pair at least, discrimination training is required to confer an odor-specific memory trace. This requires that there is at least some difference in processing between the two odors already at the beginning of the training. Therefore, as a default, there is a small yet salient difference in processing between 1-octen-3-ol and 3-octanol; this difference is ignored after nondiscriminative training (generalization), whereas it is accentuated by odor-specific reinforcement (discrimination). Given that, as the authors show, both faculties are lost in anosmic Or83b(1) mutants, this indicates an adaptive adjustment of the generalization-discrimination balance in larval Drosophila, taking place downstream of Or83b-expressing sensory neurons.

Understanding brain function is to account for how the sensory system is integrated with the organism's needs to organize behaviour. We review what is known about these processes with regard to chemosensation and chemosensory learning in Drosophila. We stress that taste and olfaction are organized rather differently. Given that, e.g., sugars are nutrients and should be eaten (irrespective of the kind of sugar) and that toxic substances should be avoided (regardless of the kind of death they eventually cause), tastants are classified into relatively few behavioural matters of concern. In contrast, what needs to be done in response to odours is less evolutionarily determined. Thus, discrimination ability is warranted between different kinds of olfactory input, as any difference between odours may potentially be or become important. Therefore, the olfactory system has a higher dimensionality than gustation, and allows for more sensory—motor flexibility to attach acquired behavioural ‘meaning’ to odours. We argue that, by and large, larval and adult Drosophila are similar in these kinds of architecture, and that additionally there are a number of similarities to vertebrates, in particular regarding the cellular architecture of the olfactory pathway, the functional slant of the taste and smell systems towards classification versus discrimination, respectively, and the higher plasticity of the olfactory sensory—motor system. From our point of view, the greatest gap in understanding smell and taste systems to date is not on the sensory side, where indeed impressive advances have been achieved; also, a satisfying account of associative odour-taste memory trace formation seems within reach. Rather, we lack an understanding as to how sensory and motor formats of processing are centrally integrated, and how adaptive motor patterns actually are selected. Such an understanding, we believe, will allow the analysis to be extended to the motivating factors of behaviour, eventually leading to a comprehensive account of those systems which make Drosophila do what Drosophila's got to do.

Why does Pavlov's dog salivate? In response to the tone, or in expectation of food? While in vertebrates behaviour can be driven by expected outcomes, it is unknown whether this is true for non-vertebrates as well. We find that, in the Drosophila larva, odour memories are expressed behaviourally only if animals can expect a positive outcome from doing so. The expected outcome of tracking down an odour is determined by comparing the value of the current situation with the value of the memory for that odour. Memory is expressed behaviourally only if the expected outcome is positive. This uncovers a hitherto unrecognized evaluative processing step between an activated memory trace and behaviour control, and argues that learned behaviour reflects the pursuit of its expected outcome. Shown in a system with a simple brain, an apparently cognitive process like representing the expected outcome of behaviour seems to be a basic feature of behaviour control.

SUMMARYAn understanding of associative learning is facilitated if it can be analyzed in a simple animal like the fruit fly Drosophila. Here, we introduce the first visual associative learning paradigm for larval Drosophila; this is remarkable as larvae have an order of magnitude fewer neurons than adult flies. Larvae were subjected to either of two reciprocal training regimes: Light+/Dark- or Light-/Dark+. Subsequently, all larvae were individually tested for their preference between Light versus Dark. The difference between training regimes was therefore exclusively which visual situation was associated with which reinforcer;differences observed during the test thus reflected exclusively associative learning. For positive reinforcement (+) we used fructose (FRU), and for negative reinforcement (-) either quinine or sodium chloride (QUI, NaCl). Under these conditions, associative learning could be reproducibly observed in both wild-type strains tested. We then compared the effectiveness of training using differential conditioning, with both positive and negative reinforcement, to that using only positive or only negative reinforcement. We found that FRU only, but neither QUI nor NaCl, was in itself effective as a reinforcer. This is the first demonstration of appetitive learning in larval Drosophila. It is now possible to investigate the behavioral and neuronal organization of appetitive visual learning in this simple and genetically easy-to-manipulate experimental system.

We addressed the question of whether the amount of individual experience determines the use of elemental or configural visual discrimination strategies in free-flying honeybees Apis mellifera. We trained bees to fly into a Y-maze to collect sucrose solution on a rewarded stimulus presented in one of the arms of the maze. Stimuli were colour disks, violet (V), green (G) or yellow (Y), which were of equal psychophysical salience for honeybees. Training followed an A+, BC+ design, followed by an AC versus BC test. Training consisted of 6 (3 A+ and 3 BC+), 20 (10 A+ and 10 BC+) or 40 (20 A+ and 20 BC+) acquisition trials. Elemental models of compound processing predict a preference for the non-trained stimulus AC while configural models predict a preference for the trained stimulus BC. Our results show that increasing the number of acquisition trials results in a change of the internal representation of stimuli. After six training trials, bees favoured an elemental strategy and preferred AC to BC during the tests. Generally, increasing the number of training trials resulted in an increase of the choice of BC. Thus, short training favoured processing of the compound as the sum of its elements (elemental account) while long training favoured its processing as being different from the sum of its elements (configural account). Additionally, we observed that the change in stimulus processing was also influenced by stimulus similarity. Colour perceptual similarity favoured configural processing with increasing experience.

Insect and mammalian olfactory systems are strikingly similar. Therefore, Drosophila can be used as a simple model for olfaction and olfactory learning. The brain of adult Drosophila, however, is still complex. We therefore chose to work on the larva with its yet simpler but adult-like olfactory system and provide evidence for olfactory learning in individually assayed Drosophila larvae. We developed a differential conditioning paradigm in which odorants are paired with positive (“+” fructose) or negative (“-” quinine or sodium chloride) gustatory reinforcers. Test performance of individuals from two treatment conditions is compared—one received odorant A with the positive reinforcer and odorant B with a negative reinforcer (A+/B-); animals from the other treatment condition were trained reciprocally (A-/B+). During test, differences in choice between A and B of individuals having undergone either A+/B- or A-/B+ training therefore indicate associative learning. We provide such evidence for both combinations of reinforcers; this was replicable across repetitions, laboratories, and experimenters. We further show that breaks improve performance, in accord with basic principles of associative learning. The present individual assay will facilitate electrophysiological studies, which necessarily use individuals. As such approaches are established for the larval neuromuscular synapse, but not in adults, an individual larval learning paradigm will serve to link behavioral levels of analysis to synaptic physiology.

We investigate olfactory memory consolidation in honeybees. Three experiments are reported that include 1024 animals in 28 experimental groups. After one pairing of odorant and sucrose reward, retention is typically nonmonotonic with a minimum 3 min after conditioning. This corresponds to the “Kamin effect” in vertebrates; the postminimum rise in retention is usually interpreted as reflecting memory consolidation. First, we test for the generality of this effect across four different odorants. The postminimum rise in retention was reproducibly observed for 1-hexanol but not for 1-octanol, limonene, or geraniol. Second, we investigate whether previous learning about the training context modulates subsequent memory consolidation. On the day before training, a reward was applied either upon placement into the future training context for 1 min, halfway during exposure or just before removal from the context. In the latter group, the 3-min minimum in retention was eliminated; thus, in that group, forward pairings of context and reward (i.e., context exposure begins before reward is applied) lead to an associative context memory that can modulate subsequent olfactory memory consolidation. Third, we found no evidence for a modulation of olfactory memory consolidation by pre-exposure to the odorant.

We use binary odorant compounds to investigate ‘blocking’ in honeybees which learn to associate an odorant (A–D) with a sucrose reward as the reinforcer (+). ‘Blocking’ means that learning about a stimulus B is reduced when trained in compound with a stimulus A that has previously been trained alone. Thus, reinforcement of B in these circumstances is not sufficient to induce learning. Such blocking is a frequently observed phenomenon in vertebrate learning and has also recently been reported in honeybee olfactory learning. To explain blocking, current models of conditioning include cognition-like concepts of attention or expectation which, consequently, seem also to apply to honeybees. Here, we first reproduce a blocking-like effect in an experimental design taken from the literature. We identify two confounding variables in that design and experimentally demonstrate their potential to support a blocking-like effect. After eliminating these confounding variables using a series of different training procedures, the blocking-like effect disappeared. Thus, convincing evidence for blocking in honeybee classical conditioning is at present lacking. This casts doubt on the applicability of cognition-like concepts to honeybees.

The question of which acquisition parameters govern long-term retention is important to an understanding of memory function. We investigate the effects of the time interval between learning trials on mediate (1 day)- and long-term (4 days) retention. In classical conditioning of the proboscis extension reflex, we train honeybees to associate an odorant with a sucrose reward using intertrial intervals of either 30 s, 1 min, 3 min, or 20 min. Intervals of 20 and 1 min result in stable retention but 3-min and 30-s intervals result in reduced retention after 4 days compared to that seen after 1 day. Thus, stability of long-term retention depends nonmonotonously on the intertrial interval. Reduced retention with 3-min intervals might be caused by a disruption of memory consolidation which is known to be especially sensitive to interference 3 min after a conditioning trial. Habituation and backward inhibitory learning are discussed as explanations for reduced retention with 30-s intervals.

To see what effects learning in one context can exercise upon subsequent learning in other contexts the relation between olfactory learning performance of honey bees,Apis melliferaL., in a restrained classical conditioning paradigm for the proboscis extension reflex and chemosensory experience under natural circumstances was investigated. The effects of food-source odours and of the hive's odour on subsequent laboratory proboscis extension performance were tested. There was a transfer effect for odours from a food source to proboscis extension reflex conditioning in the laboratory: bees extended their proboscis in response to food-source odour and showed resistance to its extinction. Hive odour had no detectable effect. Thus, the observed transfer effect for food-source odours was likely to be caused by memories established during foraging rather than by within-hive experience with the scent of harvested food. The observed transfer effect for odours from a food source thus points to a surprisingly weak role of context for the retrieval of olfactory memories established during foraging. Furthermore, it offers an experience-based explanation for variability in honey bee proboscis extension learning.

Linking a smell with an electric shock does not always have an aversive effect in flies. Can relief from pain be a pleasure? If so, noxious events should — despite their typically aversive effects — also have a ‘rewarding’ after-effect1,2,3. Through training fruitflies by using an electric shock paired with an odour, we show here that the shock can condition either avoidance of this odour or approach to it. These opposing behaviours depend on the relative timing of the shock and odour presentations during training, and indicate that a shock can act as either an aversive reinforcer or an appetitive one.

Linking a smell with an electric shock does not always have an aversive effect in flies. Can relief from pain be a pleasure? If so, noxious events should — despite their typically aversive effects — also have a ‘rewarding’ after-effect1,2,3. Through training fruitflies by using an electric shock paired with an odour, we show here that the shock can condition either avoidance of this odour or approach to it. These opposing behaviours depend on the relative timing of the shock and odour presentations during training, and indicate that a shock can act as either an aversive reinforcer or an appetitive one.

Sensory preconditioning means that reinforcement of stimulus A after unreinforced exposure to a compound AB also leads to responses to stimulus B. Here, we describe and analyze sensory preconditioning in an insect, the honeybee Apis mellifera. Using two-element odorant compounds in classical conditioning of the proboscis extension reflex, we found (i) that sensory preconditioning is not due to stimulus generalization, (ii) that paired, but not unpaired, presentation of elements supports sensory preconditioning, (iii) that simultaneous, but not sequential, exposure to the elements of the compound supports sensory preconditioning and (iv) that a single presentation of the compound yields maximal sensory preconditioning. The results are discussed with respect to configural and chain-like associative explanations for sensory preconditioning. We suggest an experience-dependent step of compound processing, establishing configural units, as an additional explanation for sensory preconditioning.

Drosophila larvae combine a numerically simple brain, a correspondingly moderate behavioral complexity, and the availability of a rich toolbox for transgenic manipulation. This makes them attractive as a study case when trying to achieve a circuit-level understanding of behavior organization. From a series of behavioral experiments, we suggest a circuitry of chemosensory processing, odor–tastant memory trace formation, and the “decision” process to behaviorally express these memory traces—or not. The model incorporates statements about the neuronal organization of innate vs. conditioned chemosensory behavior, and the types of interaction between olfactory and gustatory pathways during the establishment as well as the behavioral expression of odor–tastant memory traces. It in particular suggests that innate olfactory behavior is responsive in nature, whereas conditioned olfactory behavior is captured better when seen as an action in pursuit of its outcome. It incorporates the available neuroanatomical and behavioral data and thus should be useful as scaffold for the ongoing investigations of the chemo-behavioral system in larval Drosophila.