Search

Understanding neural activity and information representation is crucial for advancing knowledge of brain function and cognition. Neural activity, measured through techniques like electrophysiology and neuroimaging, reflects various aspects of information processing. Recent advances in deep neural networks offer new approaches to analyzing these signals using pre-trained models. However, challenges arise due to discrepancies between different neural signal modalities and the limited scale of high-quality neural data. To address these challenges, we present NeuroBind, a general representation that unifies multiple brain signal types, including EEG, fMRI, calcium imaging, and spiking data. To achieve this, we align neural signals in these image-paired neural datasets to pre-trained vision-language embeddings. Neurobind is the first model that studies different neural modalities interconnectedly and is able to leverage high-resource modality models for various neuroscience tasks. We also showed that by combining information from different neural signal modalities, NeuroBind enhances downstream performance, demonstrating the effectiveness of the complementary strengths of different neural modalities. As a result, we can leverage multiple types of neural signals mapped to the same space to improve downstream tasks, and demonstrate the complementary strengths of different neural modalities. This approach holds significant potential for advancing neuroscience research, improving AI systems, and developing neuroprosthetics and brain-computer interfaces.

Convolutional neural networks (CNNs) have been shown to be state-of-the-art models for visual cortical neurons. Cortical neurons in the primary visual cortex are sensitive to contextual information mediated by extensive horizontal and feedback connections. Standard CNNs integrate global contextual information to model contextual modulation via two mechanisms: successive convolutions and a fully connected readout layer. In this paper, we find that self-attention (SA), an implementation of non-local network mechanisms, can improve neural response predictions over parameter-matched CNNs in two key metrics: tuning curve correlation and peak tuning. We introduce peak tuning as a metric to evaluate a model's ability to capture a neuron's feature preference. We factorize networks to assess each context mechanism, revealing that information in the local receptive field is most important for modeling overall tuning, but surround information is critically necessary for characterizing the tuning peak. We find that self-attention can replace posterior spatial-integration convolutions when learned incrementally, and is further enhanced in the presence of a fully connected readout layer, suggesting that the two context mechanisms are complementary. Finally, we find that decomposing receptive field learning and contextual modulation learning in an incremental manner may be an effective and robust mechanism for learning surround-center interactions.

Photonics offers a transformative approach to artificial intelligence (AI) and neuromorphic computing by enabling low-latency, high-speed, and energy-efficient computations. However, conventional photonic tensor cores face significant challenges in constructing large-scale photonic neuromorphic networks. Here, we propose a fully integrated photonic tensor core, consisting of only two thin-film lithium niobate (TFLN) modulators, a III-V laser, and a charge-integration photoreceiver. Despite its simple architecture, it is capable of implementing an entire layer of a neural network with a computational speed of 120 GOPS, while also allowing flexible adjustment of the number of inputs (fan-in) and outputs (fan-out). Our tensor core supports rapid in-situ training with a weight update speed of 60 GHz. Furthermore, it successfully classifies (supervised learning) and clusters (unsupervised learning) 112 * 112-pixel images through in-situ training. To enable in-situ training for clustering AI tasks, we offer a solution for performing multiplications between two negative numbers., Comment: 21 pages, 6 figures

Unmanned Surface Vehicles (USVs) in the ocean environment, considering various spatiotemporal factors such as ocean currents and other energy consumption factors. The paper uses Gaussian Process Motion Planning (GPMP2), a Bayesian optimization method that has shown promising results in continuous and nonlinear motion planning algorithms. The proposed work improves GPMP2 by incorporating a new spatiotemporal factor for tracking and predicting ocean currents using a spatiotemporal Bayesian inference. The algorithm is applied to the USV path planning and is shown to optimize for smoothness, obstacle avoidance, and ocean currents in a challenging environment. The work is relevant for practical applications in ocean scenarios where optimal path planning for USVs is essential for minimizing costs and optimizing performance., Comment: 10 pages and 8 figures, submitted for IEEE Transactions on Man, systems ,and Cybernetics

The visual system evolved to process natural scenes, yet most of our understanding of the topology and function of visual cortex derives from studies using artificial stimuli. To gain deeper insights into visual processing of natural scenes, we utilized widefield calcium-imaging of primate V4 in response to many natural images, generating a large dataset of columnar-scale responses. We used this dataset to build a digital twin of V4 via deep learning, generating a detailed topographical map of natural image preferences at each cortical position. The map revealed clustered functional domains for specific classes of natural image features. These ranged from surface-related attributes like color and texture to shape-related features such as edges, curvature, and facial features. We validated the model-predicted domains with additional widefield calcium-imaging and single-cell resolution two-photon imaging. Our study illuminates the detailed topological organization and neural codes in V4 that represent natural scenes., Comment: 39 pages, 14 figures

A collective excitation of the spin structure in a magnetic insulator can transmit spin-angular momentum with negligible dissipation. This quantum of a spin wave, introduced more than nine decades ago, has always been manipulated through magnetic dipoles, (i.e., timereversal symmetry). Here, we report the experimental observation of chiral-spin transport in multiferroic BiFeO3, where the spin transport is controlled by reversing the ferroelectric polarization (i.e., spatial inversion symmetry). The ferroelectrically controlled magnons produce an unprecedented ratio of up to 18% rectification at room temperature. The spin torque that the magnons in BiFeO3 carry can be used to efficiently switch the magnetization of adja-cent magnets, with a spin-torque efficiency being comparable to the spin Hall effect in heavy metals. Utilizing such a controllable magnon generation and transmission in BiFeO3, an alloxide, energy-scalable logic is demonstrated composed of spin-orbit injection, detection, and magnetoelectric control. This observation opens a new chapter of multiferroic magnons and paves an alternative pathway towards low-dissipation nanoelectronics.

One-dimensional electron systems (1DESs) exhibit properties that are fundamentally different from higher-dimensional systems. For example, electron-electron interactions in 1DESs have been predicted to induce Tomonaga-Luttinger liquid behavior. Naturally-occurring grain boundaries in single-layer semiconducting transition metal dichalcogenides provide 1D conducting channels that have been proposed to host Tomonaga-Luttinger liquids, but charge density wave physics has also been suggested to explain their behavior. Clear identification of the electronic ground state of this system has been hampered by an inability to electrostatically gate such boundaries and thereby tune their charge carrier concentration. Here we present a scanning tunneling microscopy/spectroscopy study of gate-tunable mirror twin boundaries (MTBs) in single-layer 1H-MoSe$_2$ devices. Gating here enables STM spectroscopy to be performed for different MTB electron densities, thus allowing precise characterization of electron-electron interaction effects. Visualization of MTB electronic structure under these conditions allows unambiguous identification of collective density wave excitations having two distinct velocities, in quantitative agreement with the spin-charge separation predicted by finite-length Tomonaga-Luttinger-liquid theory., Comment: 5 figures

The interaction between various wavelike structures in screeching jets is considered via both experimental measurements and linear stability theory. Velocity snapshots of screeching jets are used to produce a reduced order model of the screech cycle via proper orthogonal decomposition. Streamwise Fourier filtering is then applied to isolate the negative and positive wavenumber components, which for the waves of interest in this jet correspond to upstream and downstream-travelling waves. A global stability analysis on an experimentally derived base flow is conducted, demonstrating a close match to the results obtained via experiment, indicating that the mechanisms considered here are well represented in a linear framework. In both analyses, three distinct wavelike structures are evident. These three waves are those first shown by Tam & Hu (1989) to be supported by a cylindrical vortex sheet. One is the Kelvin-Helmholtz mode. Another is the upstream-travelling guided jet mode that has been a topic of recent discussion. The third component, with positive phase velocity, has not previously been identified in screeching jets. We provide evidence that this downstream-travelling wave is a duct-like mode similar to that recently identified in high-subsonic jets by Towne et al. (2017). We further demonstrate that both of the latter two waves are generated by the interaction between the Kelvin-Helmholtz wavepacket and the shock cells in the flow, according to a theory first proposed in Tam & Tanna (1982). Finally, we consider the periodic spatial modulation of the coherent velocity fluctuation evident in screeching jets, and show that this modulation is the result of the superposition of the three wavelike structures, with no evidence that the shocks in the flow modulate the growth of the Kelvin-Helmholtz wavepacket., Comment: 27 pages, 10 figures. Submitted to the Journal of Fluid Mechanics

Distributed localization is essential in many robotic collective tasks such as shape formation and self-assembly.Inspired by the statistical mechanics of energy transition, this paper presents a fully distributed localization algorithm named as virtual particle exchange (VPE) localization algorithm, where each robot repetitively exchanges virtual particles (VPs) with neighbors and eventually obtains its relative position from the virtual particle (VP) amount it owns. Using custom-designed hardware and protocol, VPE localization algorithm allows robots to achieve localization using sensor readings only, avoiding direct communication with neighbors and keeping anonymity. Moreover, VPE localization algorithm determines the swarm center automatically, thereby eliminating the requirement of fixed beacons to embody the origin of coordinates. Theoretical analysis proves that the VPE localization algorithm can always converge to the same result regardless of initial state and has low asymptotic time and memory complexity. Extensive localization simulations with up to 10000 robots and experiments with 52 lowcost robots are carried out, which verify that VPE localization algorithm is scalable, accurate and robust to sensor noises. Based on the VPE localization algorithm, shape formations are further achieved in both simulations and experiments with 52 robots, illustrating that the algorithm can be directly applied to support swarm collaborative tasks.