Search

Image fusion integrates essential information from multiple images into a single composite, enhancing structures, textures, and refining imperfections. Existing methods predominantly focus on pixel-level and semantic visual features for recognition, but often overlook the deeper text-level semantic information beyond vision. Therefore, we introduce a novel fusion paradigm named image Fusion via vIsion-Language Model (FILM), for the first time, utilizing explicit textual information from source images to guide the fusion process. Specifically, FILM generates semantic prompts from images and inputs them into ChatGPT for comprehensive textual descriptions. These descriptions are fused within the textual domain and guide the visual information fusion, enhancing feature extraction and contextual understanding, directed by textual semantic information via cross-attention. FILM has shown promising results in four image fusion tasks: infrared-visible, medical, multi-exposure, and multi-focus image fusion. We also propose a vision-language dataset containing ChatGPT-generated paragraph descriptions for the eight image fusion datasets across four fusion tasks, facilitating future research in vision-language model-based image fusion. Code and dataset are available at https://github.com/Zhaozixiang1228/IF-FILM., Comment: Accepted by International Conference on Machine Learning (ICML) 2024

The observation of superconductivity in infinite-layer nickelates has attracted significant attention due to its potential as a new platform for exploring high $ \mathrm{\textit{T}}_{c} $ superconductivity. However, thus far, superconductivity has only been observed in epitaxial thin films, which limits the manipulation capabilities and modulation methods compared to two-dimensional exfoliated materials. Given the exceptionally giant strain tunability and stacking capability of freestanding membranes, separating superconducting nickelates from the as-grown substrate is a novel way to engineer the superconductivity and uncover the underlying physics. Herein, we report the synthesis of the superconducting freestanding $ \mathrm{La}_{0.8}\mathrm{Sr}_{0.2}\mathrm{Ni}\mathrm{O}_{2} $ membranes ($ \mathrm{\textit{T}}_{c}\mathrm{=}\mathrm{10.9}\;\mathrm{K} $), emphasizing the crucial roles of the interface engineering in the precursor phase film growth and the quick transfer process in achieving superconductivity. Our work offers a new versatile platform for investigating the superconductivity in nickelates, such as the pairing symmetry via constructing Josephson tunneling junctions and higher $ \mathrm{\textit{T}}_{c} $ values via high-pressure experiments., Comment: 20 pages, 9 figures

In the realm of intelligent transportation systems, accurate and reliable traffic monitoring is crucial. Traditional devices, such as cameras and lidars, face limitations in adverse weather conditions and complex traffic scenarios, prompting the need for more resilient technologies. This paper presents traffic flow monitoring method using optical fiber-based Distributed Acoustic Sensors (DAS). An innovative vehicle trajectory extraction algorithm is proposed to derive traffic flow statistics. In the processing of optical fiber waterfall diagrams, Butterworth low-pass filter and peaks location search method are employed to determine the entry position of vehicles. Subsequently, line-by-line matching algorithm is proposed to effectively track the trajectories. Experiments were conducted under both real highway and tunnel scenarios, showing that our approach not only extracts vehicle trajectories more accurately than the classical Hough and Radon transform-based methods, but also facilitates the calculation of traffic flow information using the low-cost acoustic sensors. It provides a new reliable means for traffic flow monitoring which can be integrated with existing methods., Comment: 10 pages,23 figures, references added

Bloch oscillations refer to the periodic oscillation of a wavepacket in a lattice under a constant force. Typically, the oscillation has a fundamental period that corresponds to the wavepacket traversing the first Brillouin zone once. Here we demonstrate, both theoretically and experimentally, the optical Bloch oscillations where the wavepacket must traverse the first Brillouin zone twice to complete a full cycle, resulting in a period of oscillation that is two times longer than that of usual Bloch oscillations. The unusual Bloch oscillations arise due to the band crossing of valley-Hall topological edge states at the Brillouin boundary for zigzag domain walls between two staggered honeycomb lattices with inverted on-site energy detuning, which are protected by the glide-reflection symmetry of the underlying structures. Our work sheds light on the direct detection of band crossings resulting from intrinsic symmetries that extend beyond the fundamental translational symmetry in topological systems., Comment: 4 pages, 4 figures, to appear in Phys. Rev. Lett

In an earlier paper of the authors, a general family of instances of the relative Langlands duality of Ben-Zvi-Sakellaridis-Venkatesh [BZSV] were proposed and studied in the setting of branching problems for smooth representations. In this paper, we show the numerical conjectures of [BZSV] for the local Plancherel density, as well as an application to their conjectures on global periods, for this general family of instances.

Time-Triggered Ethernet (TTEthernet) has been widely applied in many scenarios such as industrial internet, automotive electronics, and aerospace, where offline routing and scheduling for TTEthernet has been largely investigated. However, predetermined routes and schedules cannot meet the demands in some agile scenarios, such as smart factories, autonomous driving, and satellite network switching, where the transmission requests join in and leave the network frequently. Thus, we study the online joint routing and scheduling problem for TTEthernet. However, balancing efficient and effective routing and scheduling in an online environment can be quite challenging. To ensure high-quality and fast routing and scheduling, we first design a time-slot expanded graph (TSEG) to model the available resources of TTEthernet over time. The fine-grained representation of TSEG allows us to select a time slot via selecting an edge, thus transforming the scheduling problem into a simple routing problem. Next, we design a dynamic weighting method for each edge in TSEG and further propose an algorithm to co-optimize the routing and scheduling. Our scheme enhances the TTEthernet throughput by co-optimizing the routing and scheduling to eliminate potential conflicts among flow requests, as compared to existing methods. The extensive simulation results show that our scheme runs >400 times faster than standard solutions (i.e., ILP solver), while the gap is only 2% to the optimally scheduled number of flow requests. Besides, as compared to existing schemes, our method can improve the successfully scheduled number of flows by more than 18%.

This paper investigates the observational signatures of hot spots orbiting scalarized Reissner-Nordstr\"om black holes, which have been reported to possess multiple photon spheres. In contrast to the single-photon sphere case, hot spots orbiting black holes with two photon spheres produce additional image tracks in time integrated images capturing a complete orbit of hot spots. Notably, these newly observed patterns manifest as a distinct second-highest peak in temporal magnitudes when observed at low inclination angles. These findings offer promising observational probes for distinguishing black holes with multiple photon spheres from their single-photon sphere counterparts., Comment: 22 pages, 5 figures

Large Language Models (LLMs) have shown extraordinary capabilities in understanding and generating text that closely mirrors human communication. However, a primary limitation lies in the significant computational demands during training, arising from their extensive parameterization. This challenge is further intensified by the dynamic nature of the world, necessitating frequent updates to LLMs to correct outdated information or integrate new knowledge, thereby ensuring their continued relevance. Note that many applications demand continual model adjustments post-training to address deficiencies or undesirable behaviors. There is an increasing interest in efficient, lightweight methods for on-the-fly model modifications. To this end, recent years have seen a burgeoning in the techniques of knowledge editing for LLMs, which aim to efficiently modify LLMs' behaviors within specific domains while preserving overall performance across various inputs. In this paper, we first define the knowledge editing problem and then provide a comprehensive review of cutting-edge approaches. Drawing inspiration from educational and cognitive research theories, we propose a unified categorization criterion that classifies knowledge editing methods into three groups: resorting to external knowledge, merging knowledge into the model, and editing intrinsic knowledge. Furthermore, we introduce a new benchmark, KnowEdit, for a comprehensive empirical evaluation of representative knowledge editing approaches. Additionally, we provide an in-depth analysis of knowledge location, which can give a deeper understanding of the knowledge structures inherent within LLMs. Finally, we discuss several potential applications of knowledge editing, outlining its broad and impactful implications., Comment: Ongoing work; 52 pages, 282 citations; benchmark is available at https://huggingface.co/datasets/zjunlp/KnowEdit code is available at https://github.com/zjunlp/EasyEdit paper list is available at https://github.com/zjunlp/KnowledgeEditingPapers

In this paper, we investigate the thermodynamic properties of charged AdS Gauss-Bonnet black holes and the associations with the Lyapunov exponent. The chaotic features of the black holes and the isobaric heat capacity characterized by Lyapunov exponent are studied to reveal the stability of black hole phases. With the consideration of both timelike and null geodesic, we find the relationship between Lyapunov exponent and Hawking temperature can fully embody the feature of the Small/Large phase transition and the triple point even further. Then we briefly reveal the properties of Lyapunov exponent as an order parameter and explore the black hole shadow with it.

Single-image novel view synthesis is a challenging and ongoing problem that aims to generate an infinite number of consistent views from a single input image. Although significant efforts have been made to advance the quality of generated novel views, less attention has been paid to the expansion of the underlying scene representation, which is crucial to the generation of realistic novel view images. This paper proposes SinMPI, a novel method that uses an expanded multiplane image (MPI) as the 3D scene representation to significantly expand the perspective range of MPI and generate high-quality novel views from a large multiplane space. The key idea of our method is to use Stable Diffusion to generate out-of-view contents, project all scene contents into an expanded multiplane image according to depths predicted by monocular depth estimators, and then optimize the multiplane image under the supervision of pseudo multi-view data generated by a depth-aware warping and inpainting module. Both qualitative and quantitative experiments have been conducted to validate the superiority of our method to the state of the art. Our code and data are available at https://github.com/TrickyGo/SinMPI., Comment: 10 pages

The measurement of adhesion energy between nanolayers and substrates holds significant importance for the design, fabrication, and stability assessment of micro-/nanoscale devices relying on nanolayers. In this study, we propose a nanowire-supported bridging method based on an optical microscope-based nanomanipulation technique to quantitatively measure the adhesion energy between nanolayers and substrates. Using this innovative approach, we conducted adhesion energy measurements between mica nanolayers and Si substrates, revealing a value of approximately 110 J/m2. Additionally, we discuss the applicable conditions of this new method. The proposed technique allows measurements in atmospheric conditions and is, in principle, applicable to all types of nanolayers and substrates. Consequently, it holds promise as a universal method for assessing adhesion energy between nanolayers and substrates, considering environmental factors such as atmosphere and roughness.

It has been reported that the photon ring structure in black hole images produces strong and universal interferometric signatures on long interferometric baselines, holding promise for measuring black hole parameters and testing general relativity. This paper investigates the interferometric signatures of black holes with one or two photon spheres, specifically within the framework of Einstein-Maxwell-Scalar models. Notably, for black holes possessing two photon spheres, interference between light rays orbiting the inner and outer photon spheres manifests as beat signals in the visibility amplitude, deviating from the universal signatures observed in the single-photon sphere case., Comment: 19 pages, 3 figures, references added in revision

This paper is not motivated to seek innovation within the attention mechanism. Instead, it focuses on overcoming the existing trade-offs between accuracy and efficiency within the context of point cloud processing, leveraging the power of scale. Drawing inspiration from recent advances in 3D large-scale representation learning, we recognize that model performance is more influenced by scale than by intricate design. Therefore, we present Point Transformer V3 (PTv3), which prioritizes simplicity and efficiency over the accuracy of certain mechanisms that are minor to the overall performance after scaling, such as replacing the precise neighbor search by KNN with an efficient serialized neighbor mapping of point clouds organized with specific patterns. This principle enables significant scaling, expanding the receptive field from 16 to 1024 points while remaining efficient (a 3x increase in processing speed and a 10x improvement in memory efficiency compared with its predecessor, PTv2). PTv3 attains state-of-the-art results on over 20 downstream tasks that span both indoor and outdoor scenarios. Further enhanced with multi-dataset joint training, PTv3 pushes these results to a higher level., Comment: CVPR 2024, code available at Pointcept (https://github.com/Pointcept/PointTransformerV3)

This research focuses on how Large Language Models (LLMs) can help with (path) planning for mobile embodied agents such as robots, in a human-in-the-loop and interactive manner. A novel framework named LLM A*, aims to leverage the commonsense of LLMs, and the utility-optimal A* is proposed to facilitate few-shot near-optimal path planning. Prompts are used for two main purposes: 1) to provide LLMs with essential information like environments, costs, heuristics, etc.; 2) to communicate human feedback on intermediate planning results to LLMs. This approach takes human feedback on board and renders the entire planning process transparent (akin to a `white box') to humans. Moreover, it facilitates code-free path planning, thereby fostering the accessibility and inclusiveness of artificial intelligence techniques to communities less proficient in coding. Comparative analysis against A* and RL demonstrates that LLM A* exhibits greater efficiency in terms of search space and achieves paths comparable to A* while outperforming RL. The interactive nature of LLM A* also makes it a promising tool for deployment in collaborative human-robot tasks. Codes and Supplemental Materials can be found at GitHub: https://github.com/speedhawk/LLM-A-., Comment: 7 figures, 8 pages

Low earth orbit (LEO) satellite communications can provide ubiquitous and reliable services, making it an essential part of the Internet of Everything network. Beam hopping (BH) is an emerging technology for effectively addressing the issue of low resource utilization caused by the non-uniform spatio-temporal distribution of traffic demands. However, how to allocate multi-dimensional resources in a timely and efficient way for the highly dynamic LEO satellite systems remains a challenge. This paper proposes a joint beam scheduling and power optimization beam hopping (JBSPO-BH) algorithm considering the differences in the geographic distribution of sink nodes. The JBSPO-BH algorithm decouples the original problem into two sub-problems. The beam scheduling problem is modelled as a potential game, and the Nash equilibrium (NE) point is obtained as the beam scheduling strategy. Moreover, the penalty function interior point method is applied to optimize the power allocation. Simulation results show that the JBSPO-BH algorithm has low time complexity and fast convergence and achieves better performance both in throughput and fairness. Compared with greedy-based BH, greedy-based BH with the power optimization, round-robin BH, Max-SINR BH and satellite resource allocation algorithm, the throughput of the proposed algorithm is improved by 44.99%, 20.79%, 156.06%, 15.39% and 8.17%, respectively.

We introduce "ImageDream," an innovative image-prompt, multi-view diffusion model for 3D object generation. ImageDream stands out for its ability to produce 3D models of higher quality compared to existing state-of-the-art, image-conditioned methods. Our approach utilizes a canonical camera coordination for the objects in images, improving visual geometry accuracy. The model is designed with various levels of control at each block inside the diffusion model based on the input image, where global control shapes the overall object layout and local control fine-tunes the image details. The effectiveness of ImageDream is demonstrated through extensive evaluations using a standard prompt list. For more information, visit our project page at https://Image-Dream.github.io., Comment: project page: https://Image-Dream.github.io