SummaryEffective and stimulus-specific learning is essential for animals’ survival. Two major mechanisms are known to aid stimulus-specificity of associative learning. One is accurate stimulus-specific representations in neurons. The second is limited effective temporal window for the reinforcing signals to induce neuromodulation only after sensory stimuli. However, these mechanisms are often imperfect in preventing unspecific associations; different sensory stimuli can be represented by overlapping populations of neurons, and more importantly the reinforcing signals alone can induce neuromodulation even without coincident sensory-evoked neuronal activity. Here, we report a crucial neuromodulatory mechanism that counteracts both limitations and is thereby essential for stimulus specificity of learning. In Drosophila, olfactory signals are sparsely represented by cholinergic Kenyon cells (KCs), which receive dopaminergic reinforcing input. We find that KCs have numerous axo-axonic connections mediated by the muscarinic type-B receptor (mAChR-B). By using functional imaging and optogenetic approaches, we show that these axo-axonic connections suppress both odor-evoked calcium responses and dopamine-evoked cAMP signals in neighboring KCs. Strikingly, behavior experiments demonstrate that mAChR-B knockdown in KCs impairs olfactory learning by inducing undesired changes to the valence of an odor that was not associated with the reinforcer. Thus, this local neuromodulation acts in concert with sparse sensory representations and global dopaminergic modulation to achieve effective and accurate memory formation.HighlightsLateral KC axo-axonic connections are mediated by muscarinic type-B receptorKC connections suppress odor-evoked calcium responses and dopamine-evoked cAMPknockdown of the muscarinic type-B receptor impairs olfactory learningImpaired learning is due to changes to the valence of the unconditioned odor