Search

Visual search is a fundamental element of human behavior and is predominantly studied in a laboratory setting using static displays. However, real-life search is often an extended process taking place in dynamic environments. We have designed a dynamic-search task in order to incorporate the temporal dimension into visual search. Using this task, we tested how participants learn and utilize spatiotemporal regularities embedded within the environment to guide performance. Participants searched for eight instances of a target that faded in and out of a display containing similarly transient distractors. In each trial, four of the eight targets appeared in a temporally predictable fashion with one target appearing in each of four spatially separated quadrants. The other four targets were spatially and temporally unpredictable. Participants' performance was significantly better for spatiotemporally predictable compared to unpredictable targets (Experiments 1-4). The effects were reliable over different patterns of spatiotemporal predictability (Experiment 2) and primarily reflected long-term learning over trials (Experiments 3, 4), although single-trial priming effects also contributed (Experiment 4). Eye-movement recordings (Experiment 1) revealed that spatiotemporal regularities guide attention proactively and dynamically. Taken together, our results show that regularities across both space and time can guide visual search and this guidance can primarily be attributed to robust long-term representations of these regularities. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Protecting working-memory content from distracting external sensory inputs and intervening tasks is an ubiquitous demand in daily life. Here, we ask whether and how temporal expectations about external events can help mitigate effects of such interference during working-memory retention. We manipulated the temporal predictability of interfering items that occurred during the retention period of a visual working-memory task and report that temporal expectations reduce the detrimental influence of external interference on subsequent memory performance. Moreover, to determine if the protective effects of temporal expectations rely on distractor suppression or involve shielding of internal representations, we compared effects after irrelevant distractors that could be ignored vs. interrupters that required a response. Whereas distractor suppression may be sufficient to confer protection from predictable distractors, any benefits after interruption are likely to involve memory shielding. We found similar benefits of temporal expectations after both types of interference. We conclude that temporal expectations may play an important role in safeguarding behaviour based on working memory – acting through mechanisms that include the shielding of internal content from external interference.

The human brain immediately plans for prospective action when storing sensations into working memory., Working memory serves as the buffer between past sensations and future behavior, making it vital to understand not only how we encode and retain sensory information in memory but also how we plan for its upcoming use. We ask when prospective action goals emerge alongside the encoding and retention of visual information in working memory. We show that prospective action plans do not emerge gradually during memory delays but are brought into memory early, in tandem with sensory encoding. This action encoding (i) precedes a second stage of action preparation that adapts to the time of expected memory utilization, (ii) occurs even ahead of an intervening motor task, and (iii) predicts visual memory–guided behavior several seconds later. By bringing prospective action plans into working memory at an early stage, the brain creates a dual (visual-motor) memory code that can make memories more effective and robust for serving ensuing behavior.

It is believed that children have difficulties in guiding attention while facing distraction. However, developmental accounts of spatial attention rely on traditional search designs using static displays. In real life, dynamic environments can embed regularities that afford anticipation and benefit performance. We developed a dynamic visual-search task to test the ability of children to benefit from spatio-temporal regularities to detect goal-relevant targets appearing within an extended dynamic context amidst irrelevant distracting stimuli. We compared children and adults in detecting predictable vs. unpredictable targets fading in and out among competing distracting stimuli. While overall search performance was poorer in children, both groups detected more predictable targets. This effect was confined to task-relevant information. Additionally, we report how predictions are related to individual differences in attention. Altogether, our results indicate a striking capacity of prediction-led guidance towards task-relevant information in dynamic environments, refining traditional views about poor goal-driven attention in childhood.

In everyday life, attentional templates-which facilitate the perception of task-relevant sensory inputs-are often based on associations in long-term memory. We ask whether templates retrieved from memory are necessarily faithful reproductions of the encoded information or if associative-memory templates can be functionally adapted after retrieval in service of current task demands. Participants learned associations between four shapes and four colored gratings, each with a characteristic combination of color (green or pink) and orientation (left or right tilt). On each trial, observers saw one shape followed by a grating and indicated whether the pair matched the learned shape-grating association. Across experimental blocks, we manipulated the types of nonmatch (lure) gratings most often presented. In some blocks the lures were most likely to differ in color but not tilt, whereas in other blocks this was reversed. If participants functionally adapt the retrieved template such that the distinguishing information between lures and targets is prioritized, then they should overemphasize the most commonly diagnostic feature dimension within the template. We found evidence for this in the behavioral responses to the lures: participants were more accurate and faster when responding to common versus rare lures, as predicted by the functional-but not the strictly veridical-template hypothesis. This shows that templates retrieved from memory can be functionally biased to optimize task performance in a flexible, context-dependent, manner.

Probabilistic associations between stimuli afford memory templates that guide perception through proactive anticipatory mechanisms. A great deal of work has examined the behavioral consequences and human electrophysiological substrates of anticipation following probabilistic memory cues that carry spatial or temporal information to guide perception., Probabilistic associations between stimuli afford memory templates that guide perception through proactive anticipatory mechanisms. A great deal of work has examined the behavioral consequences and human electrophysiological substrates of anticipation following probabilistic memory cues that carry spatial or temporal information to guide perception. However, less is understood about the electrophysiological substrates linked to anticipating the sensory content of events based on recurring associations between successive events. Here, we demonstrate behavioral and electrophysiological signatures of using associative-memory templates to guide perception, while equating spatial and temporal anticipation (experiments 1 and 2), as well as target probability and response demands (experiment 2). By recording the electroencephalogram in the two experiments (N = 55; 24 females), we show that two markers in human electrophysiology implicated in spatial and temporal anticipation also contribute to the anticipation of perceptual identity, as follows: attenuation of alpha-band oscillations and the contingent negative variation (CNV). Together, our results show that memory-guided identity templates proactively impact perception and are associated with anticipatory states of attenuated alpha oscillations and the CNV. Furthermore, by isolating object–identity anticipation from spatial and temporal anticipation, our results suggest a role for alpha attenuation and the CNV in specific visual content anticipation beyond general changes in neural excitability or readiness. SIGNIFICANCE STATEMENT Probabilistic associations between stimuli afford memory templates that guide perception through proactive anticipatory mechanisms. The current work isolates the behavioral benefits and electrophysiological signatures of memory-guided identity-based anticipation, while equating anticipation of space, time, motor responses, and task relevance. Our results show that anticipation of the specific identity of a forthcoming percept impacts performance and is associated with states of attenuated alpha oscillations and the contingent negative variation, extending previous work implicating these neural substrates in spatial and temporal preparatory attention. Together, this work bridges fields of attention, memory, and perception, providing new insights into the neural mechanisms that support complex attentional templates.

Working memory serves as the buffer between past sensations and future behaviour, making it vital to understand not only how we encode and retain information in memory but also how we plan for its upcoming use. We ask when prospective action goals emerge alongside the encoding and retention of detailed sensory information in working memory – whether future action plans emerge gradually during memory delays, or are brought into memory early, in tandem with sensory encoding. To address this, we tracked neural planning for prospective actions in a working-memory task in which detailed visual shape information was linked to specific manual actions after a delay. We show that prospective action plans in human motor cortex emerge early alongside the selective encoding of visual information into memory. This ‘action imprinting’ (1) precedes a second stage of action preparation that adapts to the time of expected memory utilisation, (2) occurs even in face of an intervening motor task, and (3) predicts visual-memory-guided behaviour several seconds later. By bringing prospective action plans into working memory at an early stage, the brain creates a dual (visual-motor) memory code that can make memories more effective and robust for serving future behaviour.

The items on a memorized grocery list are not relevant in every aisle; for example, it is useless to search for the cabbage in the cereal aisle. It might be beneficial if one could mentally partition the list so only the relevant subset was active, so that vegetables would be activated in the produce section. In four experiments, we explored observers' abilities to partition memory searches. For example, if observers held 16 items in memory, but only eight of the items were relevant, would response times resemble a search through eight or 16 items? In Experiments 1a and 1b, observers were not faster for the partition set; however, they suffered relatively small deficits when "lures" (items from the irrelevant subset) were presented, indicating that they were aware of the partition. In Experiment 2 the partitions were based on semantic distinctions, and again, observers were unable to restrict search to the relevant items. In Experiments 3a and 3b, observers attempted to remove items from the list one trial at a time but did not speed up over the course of a block, indicating that they also could not limit their memory searches. Finally, Experiments 4a, 4b, 4c, and 4d showed that observers were able to limit their memory searches when a subset was relevant for a run of trials. Overall, observers appear to be unable or unwilling to partition memory sets from trial to trial, yet they are capable of restricting search to a memory subset that remains relevant for several trials. This pattern is consistent with a cost to switching between currently relevant memory items.

The arrangement of the contents of real-world scenes follows certain spatial rules that allow for extremely efficient visual exploration. What remains underexplored is the role different types of objects hold in a scene. In the current work, we seek to unveil an important building block of scenes-anchor objects. Anchors hold specific spatial predictions regarding the likely position of other objects in an environment. In a series of three eye tracking experiments we tested what role anchor objects occupy during visual search. In all of the experiments, participants searched through scenes for an object that was cued in the beginning of each trial. Critically, in half of the scenes a target relevant anchor was swapped for an irrelevant, albeit semantically consistent, object. We found that relevant anchor objects can guide visual search leading to faster reaction times, less scene coverage, and less time between fixating the anchor and the target. The choice of anchor objects was confirmed through an independent large image database, which allowed us to identify key attributes of anchors. Anchor objects seem to play a unique role in the spatial layout of scenes and need to be considered for understanding the efficiency of visual search in realistic stimuli.

This paper introduces the “hybrid foraging” paradigm. In typical visual search tasks, observers search for one instance of one target among distractors. In hybrid search, observers search through visual displays for one instance of any of several types of target held in memory. In foraging search, observers collect multiple instances of a single target type from visual displays. Combining these paradigms, in hybrid foraging tasks observers search visual displays for multiple instances of any of several types of target (as might be the case in searching the kitchen for dinner ingredients or an X-ray for different pathologies). In the present experiment, observers held 8–64 targets objects in memory. They viewed displays of 60–105 randomly moving photographs of objects and used the computer mouse to collect multiple targets before choosing to move to the next display. Rather than selecting at random among available targets, observers tended to collect items in runs of one target type. Reaction time (RT) data indicate searching again for the same item is more efficient than searching for any other targets, held in memory. Observers were trying to maximize collection rate. As a result, and consistent with optimal foraging theory, they tended to leave 25–33% of targets uncollected when moving to the next screen/patch. The pattern of RTs shows that while observers were collecting a target item, they had already begun searching memory and the visual display for additional targets, making the hybrid foraging task a useful way to investigate the interaction of visual and memory search.

In “hybrid” search tasks, observers hold multiple possible targets in memory while searching for those targets amongst distractor items in visual displays. Wolfe (2012) found that, if the target set is held constant over a block of trials, RTs in such tasks were a linear function of the number of items in the visual display and a linear function of the log of the number of items held in memory. However, in such tasks, the targets can become far more familiar than the distractors. Does this “familiarity” – operationalized here as the frequency and recency with which an item has appeared – influence performance in hybrid tasks In Experiment 1, we compared searches where distractors appeared with the same frequency as the targets to searches where all distractors were novel. Distractor familiarity did not have any reliable effect on search. In Experiment 2, most distractors were novel but some critical distractors were as common as the targets while others were 4× more common. Familiar distractors did not produce false alarm errors, though they did slightly increase response times (RTs). In Experiment 3, observers successfully searched for the new, unfamiliar item among distractors that, in many cases, had been seen only once before. We conclude that when the memory set is held constant for many trials, item familiarity alone does not cause observers to mistakenly confuse target with distractors.

Despite many recent technical advances, the human efficacy of naturalistic scene processing is still unparalleled. What guides attention in real world environments? How does scene context affect object search and classification? And how are the many predictions we have with regards to our visual environment structured? Here, we review the latest findings that speak to these questions, ranging from traditional psychophysics, eye movement, and neurocognitive studies in the lab to experiments in virtual reality (VR) and the real world. The growing interest and grasp of a scene's inner workings are enriching our understanding of the efficiency of scene perception and could inspire new technological and clinical advances.

In "hybrid search" tasks, such as finding items on a grocery list, one must search the scene for targets while also searching the list in memory. How is the representation of a visual item compared with the representations of items in the memory set? Predominant theories would propose a role for visual working memory (VWM) either as the site of the comparison or as a conduit between visual and memory systems. In seven experiments, we loaded VWM in different ways and found little or no effect on hybrid search performance. However, the presence of a hybrid search task did reduce the measured capacity of VWM by a constant amount regardless of the size of the memory or visual sets. These data are broadly consistent with an account in which VWM must dedicate a fixed amount of its capacity to passing visual representations to long-term memory for comparison to the items in the memory set. The data cast doubt on models in which the search template resides in VWM or where memory set item representations are moved from LTM through VWM to earlier areas for comparison to visual items.

Suppose you go to the supermarket with a shopping list of 10 items held in memory. Your shopping expedition can be seen as a combination of visual search and memory search. This is known as "hybrid search." There is a growing interest in understanding how hybrid search tasks are accomplished. We used eye tracking to examine how manipulating the number of possible targets (the memory set size [MSS]) changes how observers (Os) search. We found that dwell time on each distractor increased with MSS, suggesting a memory search was being executed each time a new distractor was fixated. Meanwhile, although the rate of refixation increased with MSS, it was not nearly enough to suggest a strategy that involves repeatedly searching visual space for subgroups of the target set. These data provide a clear demonstration that hybrid search tasks are carried out via a "one visual search, many memory searches" heuristic in which Os examine items in the visual array once with a very low rate of refixations. For each item selected, Os activate a memory search that produces logarithmic response time increases with increased MSS. Furthermore, the percentage of distractors fixated was strongly modulated by the MSS: More items in the MSS led to a higher percentage of fixated distractors. Searching for more potential targets appears to significantly alter how Os approach the task, ultimately resulting in more eye movements and longer response times.

The resolution of temporal attention is limited in a manner that makes it difficult to identify two targets in short succession. This limitation produces the phenomenon known as the attentional blink (AB), in which processing of a first target (T1) impairs identification of a second target (T2). In the AB literature, there is broad agreement that increasing the time it takes to process T1 leads to a larger AB. One might, therefore, predict that increasing the number of possible T1 identities, or target set, from 1 to 16 would lead to a larger AB. We were surprised to find that this manipulation of T1 difficulty had no influence on AB magnitude. In subsequent experiments, we found that AB magnitude interacts with T1 processing time only under certain circumstances. Specifically, when the T1 task was either well masked or had to be completed online, we found a reliable interaction between AB magnitude and the target set size. When neither of these conditions was fulfilled, there was no interaction between target set size and the AB. Previous research found that when the target set changes from trial to trial, trials with more possible targets elicited a larger AB. In the present study, the target set is held constant, reducing the demands on working memory. Nevertheless, AB magnitude still interacts with target set size, as long as the T1 task cannot be processed offline. Thus, the act of searching memory delays subsequent processing, even when the role of working memory has been minimized.

In “hybrid search” (Wolfe Psychological Science, 23(7), 698-703, 2012), observers search through visual space for any of multiple targets held in memory. With photorealistic objects as the stimuli, response times (RTs) increase linearly with the visual set size and logarithmically with the memory set size, even when over 100 items are committed to memory. It is well-established that pictures of objects are particularly easy to memorize (Brady, Konkle, Alvarez, & Oliva Proceedings of the National Academy of Sciences, 105, 14325–14329, 2008). Would hybrid-search performance be similar if the targets were words or phrases, in which word order can be important, so that the processes of memorization might be different? In Experiment 1, observers memorized 2, 4, 8, or 16 words in four different blocks. After passing a memory test, confirming their memorization of the list, the observers searched for these words in visual displays containing two to 16 words. Replicating Wolfe (Psychological Science, 23(7), 698-703, 2012), the RTs increased linearly with the visual set size and logarithmically with the length of the word list. The word lists of Experiment 1 were random. In Experiment 2, words were drawn from phrases that observers reported knowing by heart (e.g., “London Bridge is falling down”). Observers were asked to provide four phrases, ranging in length from two words to no less than 20 words (range 21–86). All words longer than two characters from the phrase, constituted the target list. Distractor words were matched for length and frequency. Even with these strongly ordered lists, the results again replicated the curvilinear function of memory set size seen in hybrid search. One might expect to find serial position effects, perhaps reducing the RTs for the first (primacy) and/or the last (recency) members of a list (Atkinson & Shiffrin, 1968; Murdock Journal of Experimental Psychology, 64, 482–488, 1962). Surprisingly, we showed no reliable effects of word order. Thus, in “London Bridge is falling down,” “London” and “down” were found no faster than “falling.”