Search

Humans experience the world through constantly changing visual stimuli, where scenes can shift and move, change in appearance, and vary in distance. The dynamic nature of visual perception is a fundamental aspect of our daily lives, yet the large majority of research on object and scene processing, particularly using fMRI, has focused on static stimuli. While studies of static image perception are attractive due to their computational simplicity, they impose a strong non-naturalistic constraint on our investigation of human vision. In contrast, dynamic visual stimuli offer a more ecologically-valid approach but present new challenges due to the interplay between spatial and temporal information, making it difficult to disentangle the representations of stable image features and motion. To overcome this limitation -- given dynamic inputs, we explicitly decouple the modeling of static image representations and motion representations in the human brain. Three results demonstrate the feasibility of this approach. First, we show that visual motion information as optical flow can be predicted (or decoded) from brain activity as measured by fMRI. Second, we show that this predicted motion can be used to realistically animate static images using a motion-conditioned video diffusion model (where the motion is driven by fMRI brain activity). Third, we show prediction in the reverse direction: existing video encoders can be fine-tuned to predict fMRI brain activity from video imagery, and can do so more effectively than image encoders. This foundational work offers a novel, extensible framework for interpreting how the human brain processes dynamic visual information.

A long standing goal in neuroscience has been to elucidate the functional organization of the brain. Within higher visual cortex, functional accounts have remained relatively coarse, focusing on regions of interest (ROIs) and taking the form of selectivity for broad categories such as faces, places, bodies, food, or words. Because the identification of such ROIs has typically relied on manually assembled stimulus sets consisting of isolated objects in non-ecological contexts, exploring functional organization without robust a priori hypotheses has been challenging. To overcome these limitations, we introduce a data-driven approach in which we synthesize images predicted to activate a given brain region using paired natural images and fMRI recordings, bypassing the need for category-specific stimuli. Our approach -- Brain Diffusion for Visual Exploration ("BrainDiVE") -- builds on recent generative methods by combining large-scale diffusion models with brain-guided image synthesis. Validating our method, we demonstrate the ability to synthesize preferred images with appropriate semantic specificity for well-characterized category-selective ROIs. We then show that BrainDiVE can characterize differences between ROIs selective for the same high-level category. Finally we identify novel functional subdivisions within these ROIs, validated with behavioral data. These results advance our understanding of the fine-grained functional organization of human visual cortex, and provide well-specified constraints for further examination of cortical organization using hypothesis-driven methods., Comment: NeurIPS 2023 (Oral). Project page: https://www.cs.cmu.edu/~afluo/BrainDiVE

Understanding the functional organization of higher visual cortex is a central focus in neuroscience. Past studies have primarily mapped the visual and semantic selectivity of neural populations using hand-selected stimuli, which may potentially bias results towards pre-existing hypotheses of visual cortex functionality. Moving beyond conventional approaches, we introduce a data-driven method that generates natural language descriptions for images predicted to maximally activate individual voxels of interest. Our method -- Semantic Captioning Using Brain Alignments ("BrainSCUBA") -- builds upon the rich embedding space learned by a contrastive vision-language model and utilizes a pre-trained large language model to generate interpretable captions. We validate our method through fine-grained voxel-level captioning across higher-order visual regions. We further perform text-conditioned image synthesis with the captions, and show that our images are semantically coherent and yield high predicted activations. Finally, to demonstrate how our method enables scientific discovery, we perform exploratory investigations on the distribution of "person" representations in the brain, and discover fine-grained semantic selectivity in body-selective areas. Unlike earlier studies that decode text, our method derives voxel-wise captions of semantic selectivity. Our results show that BrainSCUBA is a promising means for understanding functional preferences in the brain, and provides motivation for further hypothesis-driven investigation of visual cortex., Comment: ICLR 2024. Project page: https://www.cs.cmu.edu/~afluo/BrainSCUBA

The visual system is tasked with processing massive amounts of sensory information, but its computational power is constrained by limited metabolic resources. This results in the need for selective filtering of inputs so that the most relevant information is highlighted for further processing. This selectivity can be implemented slowly over the course of visual system evolution and development, by adapting the tuning properties of sensory neurons to optimally represent the stimuli that are most likely to be encountered during natural behavior. It can also be implemented more rapidly during the behavior of an individual organism, as when the brain enhances representations of target objects that are known to be relevant in a given context. Finally, selectivity can be implemented in the memory system, by adaptively re-formatting remembered information according to the demands of a particular task. In this dissertation, I will present three complementary experiments that exemplify the role of selective, efficient information processing in shaping visual system function.

Navigating through natural environments requires localizing objects along three distinct spatial axes. Information about position along the horizontal and vertical axes is available from an object's position on the retina, while position along the depth axis must be inferred based on second-order cues such as the disparity between the images cast on the two retinae. Past work has revealed that object position in two-dimensional (2D) retinotopic space is robustly represented in visual cortex and can be robustly predicted using a multivariate encoding model, in which an explicit axis is modeled for each spatial dimension. However, no study to date has used an encoding model to estimate a representation of stimulus position in depth. Here, we recorded BOLD fMRI while human subjects viewed a stereoscopic random-dot sphere at various positions along the depth (z) and the horizontal (x) axes, and the stimuli were presented across a wider range of disparities (out to ∼40 arcmin) compared to previous neuroimaging studies. In addition to performing decoding analyses for comparison to previous work, we built encoding models for depth position and for horizontal position, allowing us to directly compare encoding between these dimensions. Our results validate this method of recovering depth representations from retinotopic cortex. Furthermore, we find convergent evidence that depth is encoded most strongly in dorsal area V3A.

Increasingly, users of health and biomedical libraries need assistance with challenges they face in working with their own and others' data. Librarians have a unique opportunity to provide valuable support and assistance in data science and open science but may need to add to their expertise and skill set to have the most impact. This article describes the rationale for and development of the Medical Library Association Data Services Competency, which outlines a set of five key skills for data services and provides a course of study for gaining these skills.

A wealth of viral data sits untapped in publicly available metagenomic data sets when it might be extracted to create a usable index for the virological research community. We hypothesized that work of this complexity and scale could be done in a hackathon setting. Ten teams comprised of over 40 participants from six countries, assembled to create a crowd-sourced set of analysis and processing pipelines for a complex biological data set in a three-day event on the San Diego State University campus starting 9 January 2019. Prior to the hackathon, 141,676 metagenomic data sets from the National Center for Biotechnology Information (NCBI) Sequence Read Archive (SRA) were pre-assembled into contiguous assemblies (contigs) by NCBI staff. During the hackathon, a subset consisting of 2953 SRA data sets (approximately 55 million contigs) was selected, which were further filtered for a minimal length of 1 kb. This resulted in 4.2 million (Mio) contigs, which were aligned using BLAST against all known virus genomes, phylogenetically clustered and assigned metadata. Out of the 4.2 Mio contigs, 360,000 contigs were labeled with domains and an additional subset containing 4400 contigs was screened for virus or virus-like genes. The work yielded valuable insights into both SRA data and the cloud infrastructure required to support such efforts, revealing analysis bottlenecks and possible workarounds thereof. Mainly: (i) Conservative assemblies of SRA data improves initial analysis steps; (ii) existing bioinformatic software with weak multithreading/multicore support can be elevated by wrapper scripts to use all cores within a computing node; (iii) redesigning existing bioinformatic algorithms for a cloud infrastructure to facilitate its use for a wider audience; and (iv) a cloud infrastructure allows a diverse group of researchers to collaborate effectively. The scientific findings will be extended during a follow-up event. Here, we present the applied workflows, initial results

Gregory M Asnis,1,2 Margaret A Henderson21Department of Psychiatry and Behavioral Sciences, Albert Einstein College of Medicine, 2Anxiety and Depression Clinic, Montefiore Medical Center, Bronx, New York, NY, USAAbstract: Levomilnacipran (LVM, Fetzima®) was recently approved by the US Food and Drug Administration for the treatment of major depressive disorder. It is a unique dual neurotransmitter reuptake inhibitor. In contrast with other selective serotonin norepinephrine reuptake inhibitors, including duloxetine, venlafaxine, and desvenlafaxine, it has greater selectivity for inhibiting norepinephrine reuptake than serotonin reuptake. Our review focuses on the efficacy, safety, and tolerability data for five double-blind, placebo-controlled, short-term studies and two long-term studies. In the short-term studies, LVM was found to be more effective than placebo in reducing depression (Montgomery-Åsberg Depression Rating Scale) scores as well as improving functional impairment (Sheehan Disability Scale) scores. Long-term studies found LVM to be similarly effective but in the only placebo-controlled long-term study, LVM was not significantly superior to placebo. LVM is fairly well tolerated, with the most common adverse events being nausea, headache, dry mouth, hyperhidrosis, and constipation. Discontinuation rates were mildly increased in those being treated with LVM (9%) versus placebo (3%). Adverse events were not dose-related except for urinary hesitancy and erectile dysfunction. LVM was weight neutral, was not toxic to the liver, and did not cause clinically significant QTc prolongation. Consistent with being a predominant potentiator of norepinephrine, pulse and blood pressure were significantly elevated by LVM but rarely induced tachycardia or hypertension. LVM is a relatively safe alternative antidepressant treatment with minimal drug–drug interactions. It is the only antidepressant that has in its labeling that it is not only effe

Gregory M Asnis,1,2 Margaret A Henderson2 1Department of Psychiatry and Behavioral Sciences, Albert Einstein College of Medicine, New York, NY, USA; 2Anxiety and Depression Clinic, Montefiore Medical Center, New York, NY, USA Abstract: The EMSAM patch is a unique monoamine oxidase inhibitor (MAOI) being the only antidepressant utilizing a transdermal delivery system. This was welcomed by clinicians who hoped that EMSAM would be better tolerated than oral MAOIs and non-MAOI antidepressants, as well as being effective for treatment in a wide spectrum of depressed patients including atypical depression, bipolar depression, and refractory depression. Unfortunately, the clinical use of EMSAM has been underutilized and its potential usefulness overlooked. This article suggests that fear of possible side effects, particularly the “cheese reaction” and serotonin syndrome, are some of the main contributors to underutilization by clinicians. These risks have been significantly exaggerated with the 6 mg/day dose not even requiring a special diet. Other contributing factors leading to underutilization are reviewed such as: the lack of studies addressing many important clinical questions; inadequate data analyses; not evaluating the effect of EMSAM on comorbid psychiatric conditions, particularly anxiety disorders; lack of antidepressant comparators versus EMSAM; no dose–response relationship examined; various depressive subtypes and conditions are unexplored, eg, bipolar depression and refractory depression; poor insurance coverage for an expensive medication; as well as minimal marketing efforts and postmarketing studies. On the other hand, many potential advantages of EMSAM are not highlighted enough in the literature and by pharmaceutical companies which might have increased clinical interest and utilization of the antidepressant. For example, the advantages of EMSAM include: avoidance of swallowing issues, as can be seen with oral anti

Asnis GM, Henderson MA. Neuropsychiatr Dis Treat. 2014;10:1911–1923.On page 1921 “[…] Dye and now Mylan Pharmaceutical took over the marketing of EMSAM from Bristol-Myers Squibb” should be “Dey and now Mylan Pharmaceutical took over the marketing of EMSAM from Bristol-Myers Squibb”.On page 1921 “Both Dye and Mylan’s recent attempts at reanalysis […]” should be “Both Dey and Mylan’s recent attempts at reanalysis […]”.On page 1921 “With over 14 years left in the patent of EMSAM, [...]” should be “with 4 years left in the patent of EMSAM, [...]”. The authors apologize for these errors.Read the original article

Not Provided.