Search

Extremely large image generators offer significant transformative potential across diverse sectors. It allows users to design specific prompts to generate realistic images through some black-box APIs. However, some studies reveal that image generators are notably susceptible to attacks and generate Not Suitable For Work (NSFW) contents by manually designed toxin texts, especially imperceptible to human observers. We urgently need a multitude of universal and transferable prompts to improve the safety of image generators, especially black-box-released APIs. Nevertheless, they are constrained by labor-intensive design processes and heavily reliant on the quality of the given instructions. To achieve this, we introduce a black-box stealthy prompt attack (BSPA) that adopts a retriever to simulate attacks from API users. It can effectively harness filter scores to tune the retrieval space of sensitive words for matching the input prompts, thereby crafting stealthy prompts tailored for image generators. Significantly, this approach is model-agnostic and requires no internal access to the model's features, ensuring its applicability to a wide range of image generators. Building on BSPA, we have constructed an automated prompt tool and a comprehensive prompt attack dataset (NSFWeval). Extensive experiments demonstrate that BSPA effectively explores the security vulnerabilities in a variety of state-of-the-art available black-box models, including Stable Diffusion XL, Midjourney, and DALL-E 2/3. Furthermore, we develop a resilient text filter and offer targeted recommendations to ensure the security of image generators against prompt attacks in the future.

We propose a diffusion distillation method that achieves new state-of-the-art in one-step/few-step 1024px text-to-image generation based on SDXL. Our method combines progressive and adversarial distillation to achieve a balance between quality and mode coverage. In this paper, we discuss the theoretical analysis, discriminator design, model formulation, and training techniques. We open-source our distilled SDXL-Lightning models both as LoRA and full UNet weights.

Generative learning, recognized for its effective modeling of data distributions, offers inherent advantages in handling out-of-distribution instances, especially for enhancing robustness to adversarial attacks. Among these, diffusion classifiers, utilizing powerful diffusion models, have demonstrated superior empirical robustness. However, a comprehensive theoretical understanding of their robustness is still lacking, raising concerns about their vulnerability to stronger future attacks. In this study, we prove that diffusion classifiers possess $O(1)$ Lipschitzness, and establish their certified robustness, demonstrating their inherent resilience. To achieve non-constant Lipschitzness, thereby obtaining much tighter certified robustness, we generalize diffusion classifiers to classify Gaussian-corrupted data. This involves deriving the evidence lower bounds (ELBOs) for these distributions, approximating the likelihood using the ELBO, and calculating classification probabilities via Bayes' theorem. Experimental results show the superior certified robustness of these Noised Diffusion Classifiers (NDCs). Notably, we achieve over 80% and 70% certified robustness on CIFAR-10 under adversarial perturbations with \(\ell_2\) norms less than 0.25 and 0.5, respectively, using a single off-the-shelf diffusion model without any additional data., Comment: Accepted by NeurIPS 2024

The early stage evolution of local atomic structures in a multicomponent metallic glass during its crystallization process has been investigated via molecular dynamics simulation. It is found that the initial thermal stability and earliest stage evolution of the local atomic clusters show no strong correlation with their initial short-range orders, and this leads to an observation of a novel symmetry convergence phenomenon, which can be understood as an atomic structure manifestation of the ergodicity. Furthermore, in our system we have quantitatively proved that the crucial factor for the thermal stability against crystallization exhibited by the metallic glass, is not the total amount of icosahedral clusters, but the degree of global connectivity among them., Comment: 13 pages, 8 figures, accepted by CPB

We present an independent catalog (FRIIRGcat) of 45,241 Fanaroff-Riley Type II (FR-II) radio galaxies compiled from the Very Large Array Faint Images of the Radio Sky at Twenty-centimeters (FIRST) survey and employed the deep learning method. Among them, optical and/or infrared counterparts are identified for 41,425 FR-IIs. This catalog spans luminosities $2.63\times10^{22}\leq L_{\rm rad}\leq6.76\times10^{29}\,{\rm W}\,{\rm Hz}^{-1}$ and redshifts up to $z=5.01$. The spectroscopic classification indicates that there are 1431 low-excitation radio galaxies and 260 high-excitation radio galaxies. Among the spectroscopically identified sources, black hole masses are estimated for 4837 FR-IIs, which are in $10^{7.5}\lesssim M_{\rm BH}\lesssim 10^{9.5}$ $M_{\odot}$. Interestingly, this catalog reveals a couple of giant radio galaxies (GRGs), which are already in the existing GRG catalog, confirming the efficiency of this FR-II catalog. Furthermore, 284 new GRGs are unveiled in this new FR-II sample; they have the largest projected sizes ranging from 701 to 1209 kpc and are located at redshifts $0.31

Diffusion models trained with mean squared error loss tend to generate unrealistic samples. Current state-of-the-art models rely on classifier-free guidance to improve sample quality, yet its surprising effectiveness is not fully understood. In this paper, we show that the effectiveness of classifier-free guidance partly originates from it being a form of implicit perceptual guidance. As a result, we can directly incorporate perceptual loss in diffusion training to improve sample quality. Since the score matching objective used in diffusion training strongly resembles the denoising autoencoder objective used in unsupervised training of perceptual networks, the diffusion model itself is a perceptual network and can be used to generate meaningful perceptual loss. We propose a novel self-perceptual objective that results in diffusion models capable of generating more realistic samples. For conditional generation, our method only improves sample quality without entanglement with the conditional input and therefore does not sacrifice sample diversity. Our method can also improve sample quality for unconditional generation, which was not possible with classifier-free guidance before.

The advent of large language models (LLMs) has heightened interest in their potential for multimodal applications that integrate language and vision. This paper explores the capabilities of GPT-4V in the realms of geography, environmental science, agriculture, and urban planning by evaluating its performance across a variety of tasks. Data sources comprise satellite imagery, aerial photos, ground-level images, field images, and public datasets. The model is evaluated on a series of tasks including geo-localization, textual data extraction from maps, remote sensing image classification, visual question answering, crop type identification, disease/pest/weed recognition, chicken behavior analysis, agricultural object counting, urban planning knowledge question answering, and plan generation. The results indicate the potential of GPT-4V in geo-localization, land cover classification, visual question answering, and basic image understanding. However, there are limitations in several tasks requiring fine-grained recognition and precise counting. While zero-shot learning shows promise, performance varies across problem domains and image complexities. The work provides novel insights into GPT-4V's capabilities and limitations for real-world geospatial, environmental, agricultural, and urban planning challenges. Further research should focus on augmenting the model's knowledge and reasoning for specialized domains through expanded training. Overall, the analysis demonstrates foundational multimodal intelligence, highlighting the potential of multimodal foundation models (FMs) to advance interdisciplinary applications at the nexus of computer vision and language., Comment: 110 Pages; 61 Figures

A reliable and cost-effective interconnect technology is required for the development of hybrid pixel detectors. The interconnect technology needs to be adapted for the pitch and die sizes of the respective applications. For small-scale applications and during the ASIC and sensor development phase, interconnect technologies must also be suitable for the assembly of single-dies typically available from Multi-Project-Wafer submissions. Within the CERN EP R&D program and the AIDAinnova collaboration, innovative and scalable hybridization concepts are under development for pixel-detector applications in future colliders. This contribution presents recent results of a newly developed in-house single-die interconnection process based on Anisotropic Conductive Adhesives (ACA). The ACA interconnect technology replaces solder bumps with conductive micro-particles embedded in an epoxy layer applied as either film or paste. The electro-mechanical connection between the sensor and ASIC is achieved via thermocompression of the ACA using a flip-chip device bonder. A specific pixel-pad topology is required to enable the connection via micro-particles and create cavities into which excess epoxy can flow. This pixel-pad topology is achieved with an in-house Electroless Nickel Immersion Gold process that is also under development within the project. The ENIG and ACA processes are qualified with a variety of different ASICs, sensors, and dedicated test structures, with pad diameters ranging from 12 {\mu}m to 140 {\mu}m and pitches between 20 {\mu}m and 1.3 mm. The produced assemblies are characterized electrically, with radioactive-source exposures, and in tests with high-momentum particle beams. A focus is placed on recent optimization of the plating and interconnect processes, resulting in an improved plating uniformity and interconnect yield., Comment: Procceding to IPRD23 conference in Siena 2023

Compared with transferable untargeted attacks, transferable targeted adversarial attacks could specify the misclassification categories of adversarial samples, posing a greater threat to security-critical tasks. In the meanwhile, 3D adversarial samples, due to their potential of multi-view robustness, can more comprehensively identify weaknesses in existing deep learning systems, possessing great application value. However, the field of transferable targeted 3D adversarial attacks remains vacant. The goal of this work is to develop a more effective technique that could generate transferable targeted 3D adversarial examples, filling the gap in this field. To achieve this goal, we design a novel framework named TT3D that could rapidly reconstruct from few multi-view images into Transferable Targeted 3D textured meshes. While existing mesh-based texture optimization methods compute gradients in the high-dimensional mesh space and easily fall into local optima, leading to unsatisfactory transferability and distinct distortions, TT3D innovatively performs dual optimization towards both feature grid and Multi-layer Perceptron (MLP) parameters in the grid-based NeRF space, which significantly enhances black-box transferability while enjoying naturalness. Experimental results show that TT3D not only exhibits superior cross-model transferability but also maintains considerable adaptability across different renders and vision tasks. More importantly, we produce 3D adversarial examples with 3D printing techniques in the real world and verify their robust performance under various scenarios., Comment: Accepted by CVPR 2024

Colon cancer is expected to become the second leading cause of cancer death in the United States in 2023. Although colonoscopy is one of the most effective methods for early prevention of colon cancer, up to 30% of polyps may be missed by endoscopists, thereby increasing patients' risk of developing colon cancer. Though deep neural networks have been proven to be an effective means of enhancing the detection rate of polyps. However, the variation of polyp size brings the following problems: (1) it is difficult to design an efficient and sufficient multi-scale feature fusion structure; (2) matching polyps of different sizes with fixed-size anchor boxes is a hard challenge. These problems reduce the performance of polyp detection and also lower the model's training and detection efficiency. To address these challenges, this paper proposes a new model called YOLO-OB. Specifically, we developed a bidirectional multiscale feature fusion structure, BiSPFPN, which could enhance the feature fusion capability across different depths of a CNN. We employed the ObjectBox detection head, which used a center-based anchor-free box regression strategy that could detect polyps of different sizes on feature maps of any scale. Experiments on the public dataset SUN and the self-collected colon polyp dataset Union demonstrated that the proposed model significantly improved various performance metrics of polyp detection, especially the recall rate. Compared to the state-of-the-art results on the public dataset SUN, the proposed method achieved a 6.73% increase on recall rate from 91.5% to 98.23%. Furthermore, our YOLO-OB was able to achieve real-time polyp detection at a speed of 39 frames per second using a RTX3090 graphics card. The implementation of this paper can be found here: https://github.com/seanyan62/YOLO-OB.