Search

The advancements in large language models (LLMs) have propelled the improvement of video understanding tasks by incorporating LLMs with visual models. However, most existing LLM-based models (e.g., VideoLLaMA, VideoChat) are constrained to processing short-duration videos. Recent attempts to understand long-term videos by extracting and compressing visual features into a fixed memory size. Nevertheless, those methods leverage only visual modality to merge video tokens and overlook the correlation between visual and textual queries, leading to difficulties in effectively handling complex question-answering tasks. To address the challenges of long videos and complex prompts, we propose AdaCM$^2$, which, for the first time, introduces an adaptive cross-modality memory reduction approach to video-text alignment in an auto-regressive manner on video streams. Our extensive experiments on various video understanding tasks, such as video captioning, video question answering, and video classification, demonstrate that AdaCM$^2$ achieves state-of-the-art performance across multiple datasets while significantly reducing memory usage. Notably, it achieves a 4.5% improvement across multiple tasks in the LVU dataset with a GPU memory consumption reduction of up to 65%.

Architectural deficiencies in EPICS lead to inefficiency in the development and application of EPICS input/output controllers (IOCs). An unintrusive solution is replacing EPICS IOCs with more maintainable and flexible Python IOCs, only reusing the Channel Access (CA) protocol of EPICS. After a digression about GUI development inspired by EPICS operator interfaces (OPIs), the structural similarity between standalone GUI backends, the Mamba backend, EPICS IOCs and other server programs is analysed. By combining the caproto library and event loops like those in these server programs, the QueueIOC framework for Python IOCs is created, which has the potential to systematically replace most EPICS IOCs currently used. Examples are first given for workalikes of StreamDevice and asyn; examples for seq-like applications include monochromators, motor anti-bumping and motor multiplexing. Also shown is software to use with the ~/iocBoot convention which addresses some issues with a similar solution based on procServ, along with a workalike of procServControl. A QueueIOC-based framework for detector integration, which overcomes areaDetector's limitations in performance and architecture, is presented in an accompanying paper., Comment: 14 pages, 8 figures

Weakly-supervised methods typically guided the pixel-wise training by comparing the predictions to single-level labels containing diverse segmentation-related information at once, but struggled to represent delicate feature differences between nodule and background regions and confused incorrect information, resulting in underfitting or overfitting in the segmentation predictions. In this work, we propose a weakly-supervised network that generates multi-level labels from four-point annotation to refine diverse constraints for delicate nodule segmentation. The Distance-Similarity Fusion Prior referring to the points annotations filters out information irrelevant to nodules. The bounding box and pure foreground/background labels, generated from the point annotation, guarantee the rationality of the prediction in the arrangement of target localization and the spatial distribution of target/background regions, respectively. Our proposed network outperforms existing weakly-supervised methods on two public datasets with respect to the accuracy and robustness, improving the applicability of deep-learning based segmentation in the clinical practice of thyroid nodule diagnosis.

As the impact of climate change escalates, the global necessity to transition to sustainable energy sources becomes increasingly evident. Renewable energies have emerged as a viable solution for users, with Photovoltaic energy being a favored choice for small installations due to its reliability and efficiency. Accurate mapping of PV installations is crucial for understanding the extension of its adoption and informing energy policy. To meet this need, we introduce S3Former, designed to segment solar panels from aerial imagery and provide size and location information critical for analyzing the impact of such installations on the grid. Solar panel identification is challenging due to factors such as varying weather conditions, roof characteristics, Ground Sampling Distance variations and lack of appropriate initialization weights for optimized training. To tackle these complexities, S3Former features a Masked Attention Mask Transformer incorporating a self-supervised learning pretrained backbone. Specifically, our model leverages low-level and high-level features extracted from the backbone and incorporates an instance query mechanism incorporated on the Transformer architecture to enhance the localization of solar PV installations. We introduce a self-supervised learning phase (pretext task) to improve the initialization weights on the backbone of S3Former. We evaluated S3Former using diverse datasets, demonstrate improvement state-of-the-art models., Comment: Preprint

Estimating a prediction function is a fundamental component of many data analyses. The Super Learner ensemble, a particular implementation of stacking, has desirable theoretical properties and has been used successfully in many applications. Dimension reduction can be accomplished by using variable screening algorithms, including the lasso, within the ensemble prior to fitting other prediction algorithms. However, the performance of a Super Learner using the lasso for dimension reduction has not been fully explored in cases where the lasso is known to perform poorly. We provide empirical results that suggest that a diverse set of candidate screening algorithms should be used to protect against poor performance of any one screen, similar to the guidance for choosing a library of prediction algorithms for the Super Learner., Comment: 14 pages, 4 figures, 1 table

As climate change intensifies, the global imperative to shift towards sustainable energy sources becomes more pronounced. Photovoltaic (PV) energy is a favored choice due to its reliability and ease of installation. Accurate mapping of PV installations is crucial for understanding their adoption and informing energy policy. To meet this need, we introduce the SolarFormer, designed to segment solar panels from aerial imagery, offering insights into their location and size. However, solar panel identification in Computer Vision is intricate due to various factors like weather conditions, roof conditions, and Ground Sampling Distance (GSD) variations. To tackle these complexities, we present the SolarFormer, featuring a multi-scale Transformer encoder and a masked-attention Transformer decoder. Our model leverages low-level features and incorporates an instance query mechanism to enhance the localization of solar PV installations. We rigorously evaluated our SolarFormer using diverse datasets, including GGE (France), IGN (France), and USGS (California, USA), across different GSDs. Our extensive experiments consistently demonstrate that our model either matches or surpasses state-of-the-art models, promising enhanced solar panel segmentation for global sustainable energy initiatives., Comment: Pre-print

Correlated oxides and related heterostructures are intriguing for developing future multifunctional devices by exploiting their exotic properties, but their integration with other materials, especially on Si-based platforms, is challenging. Here, van der Waals heterostructures of La$_{0.7}$Sr$_{0.3}$MnO$_3$ (LSMO), a correlated manganite perovskite, and MoS$_2$ are demonstrated on Si substrates with multiple functions. To overcome the problems due to the incompatible growth process, technologies involving freestanding LSMO membranes and van der Waals force-mediated transfer are used to fabricate the LSMO-MoS$_2$ heterostructures. The LSMO-MoS$_2$ heterostructures exhibit a gate-tunable rectifying behavior, based on which metal-semiconductor field-effect transistors (MESFETs) with on-off ratios of over 104 can be achieved. The LSMO-MoS$_2$ heterostructures can function as photodiodes displaying considerable open-circuit voltages and photocurrents. In addition, the colossal magnetoresistance of LSMO endows the LSMO-MoS$_2$ heterostructures with an electrically tunable magnetoresponse at room temperature. This work not only proves the applicability of the LSMO-MoS$_2$ heterostructure devices on Si-based platform but also demonstrates a paradigm to create multifunctional heterostructures from materials with disparate properties.

The rapidly expanding field of metabolomics presents an invaluable resource for understanding the associations between metabolites and various diseases. However, the high dimensionality, presence of missing values, and measurement errors associated with metabolomics data can present challenges in developing reliable and reproducible methodologies for disease association studies. Therefore, there is a compelling need to develop robust statistical methods that can navigate these complexities to achieve reliable and reproducible disease association studies. In this paper, we focus on developing such a methodology with an emphasis on controlling the False Discovery Rate during the screening of mutual metabolomic signals for multiple disease outcomes. We illustrate the versatility and performance of this procedure in a variety of scenarios, dealing with missing data and measurement errors. As a specific application of this novel methodology, we target two of the most prevalent cancers among US women: breast cancer and colorectal cancer. By applying our method to the Wome's Health Initiative data, we successfully identify metabolites that are associated with either or both of these cancers, demonstrating the practical utility and potential of our method in identifying consistent risk factors and understanding shared mechanisms between diseases.

Quantum amplification is recognized as a key resource for precision measurements. However, most conventional paradigms employ an ensemble of independent particles that usually limit the performance of quantum amplification in gain, spectral linewidth, etc. Here we demonstrate a new signal amplification using cooperative 129Xe nuclear spins embedded within a feedback circuit, where the noble-gas spin coherence time is enhanced by at least one order of magnitude. Using such a technique, magnetic field can be substantially pre-enhanced by more than three orders and is in situ readout with an embedded 87Rb magnetometer. We realize an ultrahigh magnetic sensitivity of 4.0 fT/Hz$^{1/2}$ that surpasses the photon-shot noise and even below the spin-projection noise of the embedded atomic magnetometer, allowing for exciting applications including searches for dark matter with sensitivity well beyond supernova constraints. Our findings extend the physics of quantum amplification to cooperative spin systems and can be generalized to a wide variety of existing sensors, enabling a new class of cooperative quantum sensors., Comment: 7 pages, 4 figures