Search

Diffusion Models have emerged as powerful generative models for high-quality image synthesis, with many subsequent image editing techniques based on them. However, the ease of text-based image editing introduces significant risks, such as malicious editing for scams or intellectual property infringement. Previous works have attempted to safeguard images from diffusion-based editing by adding imperceptible perturbations. These methods are costly and specifically target prevalent Latent Diffusion Models (LDMs), while Pixel-domain Diffusion Models (PDMs) remain largely unexplored and robust against such attacks. Our work addresses this gap by proposing a novel attack framework, AtkPDM. AtkPDM is mainly composed of a feature representation attacking loss that exploits vulnerabilities in denoising UNets and a latent optimization strategy to enhance the naturalness of adversarial images. Extensive experiments demonstrate the effectiveness of our approach in attacking dominant PDM-based editing methods (e.g., SDEdit) while maintaining reasonable fidelity and robustness against common defense methods. Additionally, our framework is extensible to LDMs, achieving comparable performance to existing approaches.

Since the discovery of two-dimensional ferroelectric and ferromagnetic materials, the van der Waals (vdW) heterostructures constructed by ferroelectric and ferromagnetic monolayers have soon become the ideal platforms to achieve converse magnetoelectric functions at the nanoscale, namely to use electric field to control magnetization. In this Letter, by employing density functional theory calculations and dynamic simulations of atomic spin model, we study the key role of interfacial Dzyaloshinshii-Moriya interaction (DMI) in CrI$_3$-In$_2$Se$_3$ vdW heterostructures. Our work demonstrates feasible DMI torques pumped by ferroelectric switching, which can drive current-free and low-energy consumption domain wall motion. Moreover, such interfacial DMI can also significantly enlarge the Walker field in magnetic field-driven domain wall technique., Comment: 7 pages, 5 figures

The Retrieval-Augmented Language Model (RALM) has shown remarkable performance on knowledge-intensive tasks by incorporating external knowledge during inference, which mitigates the factual hallucinations inherited in large language models (LLMs). Despite these advancements, challenges persist in the implementation of RALMs, particularly concerning their reliability and traceability. To be specific, the irrelevant document retrieval may result in unhelpful response generation or even deteriorate the performance of LLMs, while the lack of proper citations in generated outputs complicates efforts to verify the trustworthiness of the models. To this end, we propose a novel self-reasoning framework aimed at improving the reliability and traceability of RALMs, whose core idea is to leverage reasoning trajectories generated by the LLM itself. The framework involves constructing self-reason trajectories with three processes: a relevance-aware process, an evidence-aware selective process, and a trajectory analysis process. We have evaluated our framework across four public datasets (two short-form QA datasets, one long-form QA dataset, and one fact verification dataset) to demonstrate the superiority of our method, which can outperform existing state-of-the-art models and can achieve comparable performance with GPT-4, while only using 2,000 training samples., Comment: AAAI 2025 (main conference)

In this work, we investigate a static spherically symmetric system in which Einstein gravity is minimally coupled with a self-interacting complex scalar field and a nonlinear electromagnetic field, referred to as Hayward axion stars. Employing numerical methods, we find that it essentially describes axion stars with the magnetic charge. In the absence of magnetic charge and with only the scalar field present, the system reduces to axion stars. We discover that when the magnetic charge $q$ exceeds a critical value, extreme solutions with frequencies $\omega$ approaching zero can be found and the critical horizon emerges. Within this horizon, the scalar field and energy density are highly concentrated and decrease precipitously at its boundary. The time component of the metric function approaches zero within this region, indicating that gravity is extremely intense, and time nearly ceases to flow. To an observer at infinity, the star appears to be frozen, hence we refer to these extreme solutions exhibiting a critical horizon as Hayward axion frozen stars. Furthermore, it is important to note that as $\omega \rightarrow 0$, the mass of the Hayward axion frozen star becomes independent of the decay constant and is only determined by the magnetic charge. Additionally, we find that the frozen star solutions possess two light rings. With an increase in the magnetic charge, these light rings move outward, while changes in the decay constant have little effect on their positions., Comment: 26 pages, 10 figures

In this paper we study Einstein-bumblebee gravity theory minimally coupled with external matter -- a phantom/non-phantom(conventional) scalar field, and derive a series of hairy solutions -- bumblebee-phantom(BP) and BP-dS/AdS black hole solutions, regular Ellis-bumblebee-phantom (EBP) and BP-AdS wormholes, etc. We first find that the Lorentz violation (LV) effect can change the so called black hole no-hair theorem and these scalar fields can give a hair to a black hole. If LV coupling constant $\ell>-1$, the phantom field is admissible and the conventional scalar field is forbidden; if $\ell<-1$, the phantom field is forbidden and the conventional scalar field is admissible. By defining the Killing potential $\omega^{ab}$, we study the Smarr formula and the first law for the BP black hole, find that the appearance of LV can improve the structure of these phantom hairy black holes -- the conventional Smarr formula and the first law of black hole thermodynamics still hold; but for no LV case, i.e., the regular phantom black hole reported in [Phys. Rev. Lett. {\bf96}, 251101], the first law cannot be constructed at all. When the bumblebee potential is linear, we find that the phantom potential and the Lagrange-multiplier $\lambda$ behave as a cosmological constant $\Lambda$., Comment: 22 pages, 4 tables, 1 figure, eight pages added

Most existing Low-light Image Enhancement (LLIE) methods either directly map Low-Light (LL) to Normal-Light (NL) images or use semantic or illumination maps as guides. However, the ill-posed nature of LLIE and the difficulty of semantic retrieval from impaired inputs limit these methods, especially in extremely low-light conditions. To address this issue, we present a new LLIE network via Generative LAtent feature based codebook REtrieval (GLARE), in which the codebook prior is derived from undegraded NL images using a Vector Quantization (VQ) strategy. More importantly, we develop a generative Invertible Latent Normalizing Flow (I-LNF) module to align the LL feature distribution to NL latent representations, guaranteeing the correct code retrieval in the codebook. In addition, a novel Adaptive Feature Transformation (AFT) module, featuring an adjustable function for users and comprising an Adaptive Mix-up Block (AMB) along with a dual-decoder architecture, is devised to further enhance fidelity while preserving the realistic details provided by codebook prior. Extensive experiments confirm the superior performance of GLARE on various benchmark datasets and real-world data. Its effectiveness as a preprocessing tool in low-light object detection tasks further validates GLARE for high-level vision applications. Code is released at https://github.com/LowLevelAI/GLARE., Comment: Accepted by ECCV 2024

The asymmetry in hard X-ray (HXR) emission at the footpoints (FPs) of flare loops is a ubiquitous feature closely associated with nonthermal electron transport. We analyze the asymmetric HXR radiation at two flare ribbons which is thermal-dominated during a long-duration C4.4 flare that occurred on March 20, 2023, combining multi-view and multi-waveband observations from the ASO-S, SolO, and SDO spacecraft. We find that the H I Ly$\alpha$ emission captures similar features to the He II $\lambda$304 in both light curve and spatio-temporal evolution of a pair of conjugate flare ribbons. The spectra and imaging analysis of the HXR emission, detected by STIX in 4-18 keV, reveal that the two-ribbon flare radiation is thermal dominated by over 95%, and the radiation source mainly concentrates on the northern ribbon, leading to an asymmetric distribution. To understand the underlying reasons for the HXR radiation asymmetry, we extrapolate the magnetic field within the active region using the NLFFF model. For 78% of the magnetic field lines starting from the northern flare ribbon, their lengths from the loop-tops (LTs) to the northern FPs are shorter than those to the southern FPs. For 62% of the field lines, their magnetic field strengths at the southern FPs exceed those at the northern FPs. In addition, considering the larger density, $\approx1.0\times10^{10}$ cm$^{-3}$, of the low-lying flare loops (< 32 Mm), we find the shorter path from the LT to the northern FP enables more electrons to reach the northern FP more easily after collisions with the surrounding plasma. Therefore, in this thermal-dominated C-class flare, the asymmetric location of the flare LT relative to its two FPs plays a dominant role in the HXR radiation asymmetry, while such asymmetry is also slightly influenced by the magnetic mirror effect resulting in larger HXR radiation at the FPs with weaker magnetic strength., Comment: 21 pages, 7 figures, accepted by Solar Physics. Comments welcome

Recent advancements in artificial intelligence (AI) have precipitated significant breakthroughs in healthcare, particularly in refining diagnostic procedures. However, previous studies have often been constrained to limited functionalities. This study introduces MiniGPT-Med, a vision-language model derived from large-scale language models and tailored for medical applications. MiniGPT-Med demonstrates remarkable versatility across various imaging modalities, including X-rays, CT scans, and MRIs, enhancing its utility. The model is capable of performing tasks such as medical report generation, visual question answering (VQA), and disease identification within medical imagery. Its integrated processing of both image and textual clinical data markedly improves diagnostic accuracy. Our empirical assessments confirm MiniGPT-Med's superior performance in disease grounding, medical report generation, and VQA benchmarks, representing a significant step towards reducing the gap in assisting radiology practice. Furthermore, it achieves state-of-the-art performance on medical report generation, higher than the previous best model by 19\% accuracy. MiniGPT-Med promises to become a general interface for radiology diagnoses, enhancing diagnostic efficiency across a wide range of medical imaging applications.

Diffusion models have marked a significant breakthrough in the synthesis of semantically coherent images. However, their extensive noise estimation networks and the iterative generation process limit their wider application, particularly on resource-constrained platforms like mobile devices. Existing post-training quantization (PTQ) methods have managed to compress diffusion models to low precision. Nevertheless, due to the iterative nature of diffusion models, quantization errors tend to accumulate throughout the generation process. This accumulation of error becomes particularly problematic in low-precision scenarios, leading to significant distortions in the generated images. We attribute this accumulation issue to two main causes: error propagation and exposure bias. To address these problems, we propose a timestep-aware correction method for quantized diffusion model, which dynamically corrects the quantization error. By leveraging the proposed method in low-precision diffusion models, substantial enhancement of output quality could be achieved with only negligible computation overhead. Extensive experiments underscore our method's effectiveness and generalizability. By employing the proposed correction strategy, we achieve state-of-the-art (SOTA) results on low-precision models., Comment: ECCV 2024

In this paper, we examine the thermodynamics of heavy quarkonium in the spinning black hole background. Specifically, we investigate the effect of angular momentum on the interquark distance, free energy, binding energy, entropy, entropic force, and internal energy of heavy quarkonium from the thermodynamic relationship. Our findings indicate that the angular momentum reduces the maximum value of interquark distance, suggesting that it promotes the dissociation of quarkonium. Additionally, we observe that the angular momentum suppresses free energy. From the results of binding energy, the angular momentum favors the melting of meson into a free quark and antiquark. Moreover, the results show that angular momentum increases the entropy and entropic force, thus accelerates the dissociation of quarkonium. The angular momentum increases the internal energy at large interquark distance. Finally, we find that the angular momentum has a more pronounced effect on quarkonium when the axis of quark pair $Q\overline{Q}$ is transverse to the direction of angular momentum., Comment: 21 pages, 7 figures, to be published in Physical Review D

Leveraging Large Language Models' remarkable proficiency in text-based tasks, recent works on Multi-modal LLMs (MLLMs) extend them to other modalities like vision and audio. However, the progress in these directions has been mostly focused on tasks that only require a coarse-grained understanding of the audio-visual semantics. We present Meerkat, an audio-visual LLM equipped with a fine-grained understanding of image and audio both spatially and temporally. With a new modality alignment module based on optimal transport and a cross-attention module that enforces audio-visual consistency, Meerkat can tackle challenging tasks such as audio referred image grounding, image guided audio temporal localization, and audio-visual fact-checking. Moreover, we carefully curate a large dataset AVFIT that comprises 3M instruction tuning samples collected from open-source datasets, and introduce MeerkatBench that unifies five challenging audio-visual tasks. We achieve state-of-the-art performance on all these downstream tasks with a relative improvement of up to 37.12%., Comment: Accepted at ECCV 2024

This paper studies the rate-distortion-perception (RDP) tradeoff for a Gaussian vector source coding problem where the goal is to compress the multi-component source subject to distortion and perception constraints. The purpose of imposing a perception constraint is to ensure visually pleasing reconstructions. This paper studies this RDP setting with either the Kullback-Leibler (KL) divergence or Wasserstein-2 metric as the perception loss function, and shows that for Gaussian vector sources, jointly Gaussian reconstructions are optimal. We further demonstrate that the optimal tradeoff can be expressed as an optimization problem, which can be explicitly solved. An interesting property of the optimal solution is as follows. Without the perception constraint, the traditional reverse water-filling solution for characterizing the rate-distortion (RD) tradeoff of a Gaussian vector source states that the optimal rate allocated to each component depends on a constant, called the water-level. If the variance of a specific component is below the water-level, it is assigned a {zero} compression rate. However, with active distortion and perception constraints, we show that the optimal rates allocated to the different components are always {positive}. Moreover, the water-levels that determine the optimal rate allocation for different components are unequal. We further treat the special case of perceptually perfect reconstruction and study its RDP function in the high-distortion and low-distortion regimes to obtain insight to the structure of the optimal solution.

Background The incidence of stroke places a heavy burden on both society and individuals. Activity is closely related to cardiovascular health. This study aimed to investigate the relationship between the varying domains of PA, like occupation-related Physical Activity (OPA), transportation-related Physical Activity (TPA), leisure-time Physical Activity (LTPA), and Sedentary Activity (SA) with stroke. Methods Our analysis included 30,400 participants aged 20+ years from 2007 to 2018 National Health and Nutrition Examination Survey (NHANES). Stroke was identified based on the participant's self-reported diagnoses from previous medical consultations, and PA and SA were self-reported. Multivariable logistic and restricted cubic spline models were used to assess the associations. Results Participants achieving PA guidelines (performing PA more than 150 min/week) were 35.7% less likely to have a stroke based on both the total PA (odds ratio [OR] 0.643, 95% confidence interval [CI] 0.523-0.790) and LTPA (OR 0.643, 95% CI 0.514-0.805), while OPA or TPA did not demonstrate lower stroke risk. Furthermore, participants with less than 7.5 h/day SA levels were 21.6% (OR 0.784, 95% CI 0.665-0.925) less likely to have a stroke. The intensities of total PA and LTPA exhibited nonlinear U-shaped associations with stroke risk. In contrast, those of OPA and TPA showed negative linear associations, while SA intensities were positively linearly correlated with stroke risk. Conclusions LTPA, but not OPA or TPA, was associated with a lower risk of stroke at any amount, suggesting that significant cardiovascular health would benefit from increased PA. Additionally, the positive association between SA and stroke indicated that prolonged sitting was detrimental to cardiovascular health. Overall, increased PA within a reasonable range reduces the risk of stroke, while increased SA elevates it.

Sympathetic eruptions of solar prominences have been studied for decades, however, it is usually difficult to identify their causal links. Here we present two failed prominence eruptions on 26 October 2022 and explore their connections. Using stereoscopic observations, the south prominence (PRO-S) erupts with untwisting motions, flare ribbons occur underneath, and new connections are formed during the eruption. The north prominence (PRO-N) rises up along with PRO-S, and its upper part disappears due to catastrophic mass draining along an elongated structure after PRO-S failed eruption. We suggest that the eruption of PRO-S initiates due to a kink instability, further rises up, and fails to erupt due to reconnection with surrounding fields. The elongated structure connecting PRO-N overlies PRO-S, which causes the rising up of PRO-N along with PRO-S and mass drainage after PRO-S eruption. This study suggests that a prominence may end its life through mass drainage forced by an eruption underneath., Comment: 15 pages, 7 figures, has been accepted by Solar Physics

This survey articles focuses on emerging connections between the fields of machine learning and data compression. While fundamental limits of classical (lossy) data compression are established using rate-distortion theory, the connections to machine learning have resulted in new theoretical analysis and application areas. We survey recent works on task-based and goal-oriented compression, the rate-distortion-perception theory and compression for estimation and inference. Deep learning based approaches also provide natural data-driven algorithmic approaches to compression. We survey recent works on applying deep learning techniques to task-based or goal-oriented compression, as well as image and video compression. We also discuss the potential use of large language models for text compression. We finally provide some directions for future research in this promising field., Comment: arXiv admin note: text overlap with arXiv:2002.04290

This paper proposes ConsistDreamer - a novel framework that lifts 2D diffusion models with 3D awareness and 3D consistency, thus enabling high-fidelity instruction-guided scene editing. To overcome the fundamental limitation of missing 3D consistency in 2D diffusion models, our key insight is to introduce three synergetic strategies that augment the input of the 2D diffusion model to become 3D-aware and to explicitly enforce 3D consistency during the training process. Specifically, we design surrounding views as context-rich input for the 2D diffusion model, and generate 3D-consistent, structured noise instead of image-independent noise. Moreover, we introduce self-supervised consistency-enforcing training within the per-scene editing procedure. Extensive evaluation shows that our ConsistDreamer achieves state-of-the-art performance for instruction-guided scene editing across various scenes and editing instructions, particularly in complicated large-scale indoor scenes from ScanNet++, with significantly improved sharpness and fine-grained textures. Notably, ConsistDreamer stands as the first work capable of successfully editing complex (e.g., plaid/checkered) patterns. Our project page is at immortalco.github.io/ConsistDreamer., Comment: CVPR 2024

This paper proposes Instruct 4D-to-4D that achieves 4D awareness and spatial-temporal consistency for 2D diffusion models to generate high-quality instruction-guided dynamic scene editing results. Traditional applications of 2D diffusion models in dynamic scene editing often result in inconsistency, primarily due to their inherent frame-by-frame editing methodology. Addressing the complexities of extending instruction-guided editing to 4D, our key insight is to treat a 4D scene as a pseudo-3D scene, decoupled into two sub-problems: achieving temporal consistency in video editing and applying these edits to the pseudo-3D scene. Following this, we first enhance the Instruct-Pix2Pix (IP2P) model with an anchor-aware attention module for batch processing and consistent editing. Additionally, we integrate optical flow-guided appearance propagation in a sliding window fashion for more precise frame-to-frame editing and incorporate depth-based projection to manage the extensive data of pseudo-3D scenes, followed by iterative editing to achieve convergence. We extensively evaluate our approach in various scenes and editing instructions, and demonstrate that it achieves spatially and temporally consistent editing results, with significantly enhanced detail and sharpness over the prior art. Notably, Instruct 4D-to-4D is general and applicable to both monocular and challenging multi-camera scenes. Code and more results are available at immortalco.github.io/Instruct-4D-to-4D., Comment: CVPR 2024

Efficient electric manipulation of valley degrees of freedom is critical and challenging for the advancement of valley-based information science and technology. We put forth an electrical scheme, based on a two-band Dirac model, that can switch the fully valley-polarized photocurrent between K and K' valleys using the circular bulk electro-photovoltaic effect. This is accomplished by applying an out-of-plane electric field to the two-dimensional valley materials, which enables continuous tuning of the Berry curvature and its sign flip. We found that the switch of the fully valley-polarized photocurrent is directly tied to the sign change of Berry curvature, which accompanies a topological phase transition, for instance, the quantum spin Hall effect and the quantum valley Hall effect. This scheme has been confirmed in monolayer BiAsI2 and germanene through first-principles calculations. Our paper offers a promising strategy for the development of a volatile valley-addressable memory device and could inspire further research in this area.

We present the new concept of photonic alloy as a non-periodic topological material. By mixing non-magnetized and magnetized rods in a non-periodic 2D photonic crystal configuration, we realized photonic alloys in the microwave regime. Our experimental findings reveal that the photonic alloy sustains non-reciprocal chiral edge states (CESs) even at very low concentration of magnetized rods. The non-trivial topology and the associated edge states of these non-periodic systems can be characterized by the winding of the reflection phase. Our results indicate that the threshold concentrations for the investigated system within the first non-trivial band gap to exhibit topological behavior approach zero in the thermodynamic limit for substitutional alloys, while the threshold remains non-zero for interstitial alloys. At low concentration, the system exhibits an inhomogeneous structure characterized by isolated patches of non-percolating magnetic domains that are spaced far apart within a topologically trivial photonic crystal. Surprisingly, the system manifests CESs despite a local breakdown of time-reversal symmetry rather than a global one. Photonic alloys represent a new category of disordered topological materials, offering exciting opportunities for exploring topological materials with adjustable gaps., Comment: 6 pages, 4 figures