Search

Large Vision-Language Models (LVLMs) demonstrate impressive capabilities in generating detailed and coherent responses from visual inputs. However, they are prone to generate hallucinations due to an over-reliance on language priors. To address this issue, we investigate the language priors in LVLMs and make two key observations: (1) Even when predicting the tokens associated with image-related part-of-speech (POS), models increasingly rely on linguistic priors as the token sequences grow, thereby amplifying hallucinations. (2) Methods that directly calibrate LVLM's output distribution to mitigate language priors can lead to a degradation in text quality or even exacerbate hallucinations. Based on these findings, we propose a novel method, Summary-Guided Decoding (SGD). This method naturally encourages the model to focus more on image information by reducing the text context through summaries, while controlling only the image-related POS tokens to maintain text quality. Through experiments, we demonstrate that SGD achieves state-of-the-art performance on object hallucination benchmarks. Furthermore, in terms of the trade-off between precision and recall, SGD achieves Pareto optimality among the existing methods. Lastly, we observe that although existing methods struggle to balance the reduction of object hallucinations with maintaining text quality, SGD demonstrates robustness in handling this challenge.

We study the phenomenology of the charged Higgs boson, $H^\pm$,appearing in the fermionic dark matter model mediated by the dark $Z$ boson. This model is in favor of the light dark $Z$ boson, $Z'$, and the light additional neutral Higgs boson, $h$. We find that $H^\pm \to W^\pm h$ and the $H^\pm \to W^\pm Z'$ are dominant decay channels. Thus the promising final states are trilepton signals, $e \mu \mu$ or $\mu \mu \mu$ following $Z' \to \mu^+ \mu^-$ decays and leptonic decays of the $W^\pm$ boson. The charged Higgs boson will be produced from the top quark decays $t \to b H^\pm$ following $t \bar{t}$ production, if $H^\pm$ is light. Whereas $H^\pm$ is heavier than the top quark, the dominant production processes are associated productions with either $Z'$ or $h$, $pp \to W^\star \to H^\pm h$ and $pp \to W^\star \to H^\pm Z'$. We explore the discovery potential of the charged Higgs boson at the LHC. We also discuss the implications of dark matter in relation with the charged Higgs phenomenology., Comment: 12 pages, 4 figures

The Mapper algorithm is a popular tool for visualization and data exploration in topological data analysis. We investigate an inverse problem for the Mapper algorithm: Given a dataset $X$ and a graph $G$, does there exist a set of Mapper parameters such that the output Mapper graph of $X$ is isomorphic to $G$? We provide constructions that affirmatively answer this question. Our results demonstrate that it is possible to engineer Mapper parameters to generate a desired graph., Comment: 13 pages, 4 figures

Location-based services offer immense utility, but also pose significant privacy risks. In response, we propose LocPIR, a novel framework using homomorphic encryption (HE), specifically the TFHE scheme, to preserve user location privacy when retrieving data from public clouds. Our system employs TFHE's expertise in non-polynomial evaluations, crucial for comparison operations. LocPIR showcases minimal client-server interaction, reduced memory overhead, and efficient throughput. Performance tests confirm its computational speed, making it a viable solution for practical scenarios, demonstrated via application to a COVID-19 alert model. Thus, LocPIR effectively addresses privacy concerns in location-based services, enabling secure data sharing from the public cloud., Comment: Accepted at the IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS) 2024

Large Vision-Language Models (LVLMs) have demonstrated outstanding performance across various multimodal tasks. However, they suffer from a problem known as language prior, where responses are generated based solely on textual patterns while disregarding image information. Addressing the issue of language prior is crucial, as it can lead to undesirable biases or hallucinations when dealing with images that are out of training distribution. Despite its importance, current methods for accurately measuring language priors in LVLMs are poorly studied. Although existing benchmarks based on counterfactual or out-of-distribution images can partially be used to measure language priors, they fail to disentangle language priors from other confounding factors. To this end, we propose a new benchmark called VLind-Bench, which is the first benchmark specifically designed to measure the language priors, or blindness, of LVLMs. It not only includes tests on counterfactual images to assess language priors but also involves a series of tests to evaluate more basic capabilities such as commonsense knowledge, visual perception, and commonsense biases. For each instance in our benchmark, we ensure that all these basic tests are passed before evaluating the language priors, thereby minimizing the influence of other factors on the assessment. The evaluation and analysis of recent LVLMs in our benchmark reveal that almost all models exhibit a significant reliance on language priors, presenting a strong challenge in the field.

We study the expected sensitivity to the mixing between sterile and tau neutrinos directly from the tau neutrino disappearance in the high-energy fixed target experiment. Here, the beam energy is large enough to produce tau neutrinos at the target with large luminosity. During their propagation to the detector, tau neutrinos may oscillate into sterile neutrinos. By examining the energy spectrum of the observed tau neutrino events, we can probe the mixing between sterile and tau neutrinos directly. In this paper, we consider Scattering and Neutrino Detector (SND) at SHiP experiment as a showcase, which uses 400 GeV protons from SPS at CERN, and expect to observe 7,300 tau and anti-tau neutrinos from the $2\times 10^{20}$ POT for 5 years operation. Assuming the uncertainty of 10\%, we find the sensitivity $|U_{\tau 4}|^2 \sim 0.08$\, (90\% CL) for $\Delta m_{41}^2 \sim 500\ \mathrm{eV}^2$ with 10\% background to the signal. We also consider a far SND at the end of the SHiP Hidden Sector Decay Spectrometer (HSDS), in which case the sensitivity would be enhanced to $|U_{\tau 4}|^2 \sim 0.02$. Away from this mass, the sensitivity becomes lower than $|U_{\tau 4}|^2 \sim 0.15$ for $\Delta m_{41}^2 \lesssim 100\ \mathrm{eV}^2$ or $\Delta m_{41}^2\gtrsim 10^4 \mathrm{eV}^2$., Comment: 14 pages, 8 figures

Synaptic vesicle transport by motor proteins along microtubules is a crucial active process underlying neuronal communication. It is known that microtubules are destabilized by tau-hyperphosphorylation, which causes tau proteins to detach from microtubules and form neurofibril tangles. However, how tau-phosphorylation affects transport dynamics of motor proteins on the microtubule remains unknown. Here, we discover that long-distance unidirectional motion of vesicle-motor protein multiplexes (VMPMs) in living cells is suppressed under tau-hyperphosphorylation, with the consequent loss of fast vesicle-transport along the microtubule. The VMPMs in hyperphosphorylated cells exhibit seemingly bidirectional random motion, with dynamic properties far different from VMPM motion in normal cells. We establish a parsimonious physicochemical model of VMPM's active motion that provides a unified, quantitative explanation and predictions for our experimental results. Our analysis reveals that, under hyperphosphorylation conditions, motor-protein-multiplexes have both static and dynamic motility fluctuations. The loss of the fast vesicle-transport along the microtubule can be a mechanism of neurodegenerative disorders associated with tau-hyperphosphorylation.

The sense of touch is an essential ability for skillfully performing a variety of tasks, providing the capacity to search and manipulate objects without relying on visual information. Extensive research has been conducted over time to apply these human tactile abilities to robots. In this paper, we introduce a multi-finger robot system designed to search for and manipulate objects using the sense of touch without relying on visual information. Randomly located target objects are searched using tactile sensors, and the objects are manipulated for tasks that mimic daily-life. The objective of the study is to endow robots with human-like tactile capabilities. To achieve this, binary tactile sensors are implemented on one side of the robot hand to minimize the Sim2Real gap. Training the policy through reinforcement learning in simulation and transferring the trained policy to the real environment, we demonstrate that object search and manipulation using tactile sensors is possible even in an environment without vision information. In addition, an ablation study was conducted to analyze the effect of tactile information on manipulative tasks. Our project page is available at https://lee-kangwon.github.io/dextouch/, Comment: Project page: https://lee-kangwon.github.io/dextouch/

Integrated computational materials engineering (ICME) has significantly enhanced the systemic analysis of the relationship between microstructure and material properties, paving the way for the development of high-performance materials. However, analyzing microstructure-sensitive material behavior remains challenging due to the scarcity of three-dimensional (3D) microstructure datasets. Moreover, this challenge is amplified if the microstructure is anisotropic, as this results in anisotropic material properties as well. In this paper, we present a framework for reconstruction of anisotropic microstructures solely based on two-dimensional (2D) micrographs using conditional diffusion-based generative models (DGMs). The proposed framework involves spatial connection of multiple 2D conditional DGMs, each trained to generate 2D microstructure samples for three different orthogonal planes. The connected multiple reverse diffusion processes then enable effective modeling of a Markov chain for transforming noise into a 3D microstructure sample. Furthermore, a modified harmonized sampling is employed to enhance the sample quality while preserving the spatial connection between the slices of anisotropic microstructure samples in 3D space. To validate the proposed framework, the 2D-to-3D reconstructed anisotropic microstructure samples are evaluated in terms of both the spatial correlation function and the physical material behavior. The results demonstrate that the framework is capable of reproducing not only the statistical distribution of material phases but also the material properties in 3D space. This highlights the potential application of the proposed 2D-to-3D reconstruction framework in establishing microstructure-property linkages, which could aid high-throughput material design for future studies

Executing contact-rich manipulation tasks necessitates the fusion of tactile and visual feedback. However, the distinct nature of these modalities poses significant challenges. In this paper, we introduce a system that leverages visual and tactile sensory inputs to enable dexterous in-hand manipulation. Specifically, we propose Robot Synesthesia, a novel point cloud-based tactile representation inspired by human tactile-visual synesthesia. This approach allows for the simultaneous and seamless integration of both sensory inputs, offering richer spatial information and facilitating better reasoning about robot actions. The method, trained in a simulated environment and then deployed to a real robot, is applicable to various in-hand object rotation tasks. Comprehensive ablations are performed on how the integration of vision and touch can improve reinforcement learning and Sim2Real performance. Our project page is available at https://yingyuan0414.github.io/visuotactile/ ., Comment: Project page: https://yingyuan0414.github.io/visuotactile/

Conversational search aims to retrieve passages containing essential information to answer queries in a multi-turn conversation. In conversational search, reformulating context-dependent conversational queries into stand-alone forms is imperative to effectively utilize off-the-shelf retrievers. Previous methodologies for conversational query reformulation frequently depend on human-annotated rewrites. However, these manually crafted queries often result in sub-optimal retrieval performance and require high collection costs. To address these challenges, we propose Iterative Conversational Query Reformulation (IterCQR), a methodology that conducts query reformulation without relying on human rewrites. IterCQR iteratively trains the conversational query reformulation (CQR) model by directly leveraging information retrieval (IR) signals as a reward. Our IterCQR training guides the CQR model such that generated queries contain necessary information from the previous dialogue context. Our proposed method shows state-of-the-art performance on two widely-used datasets, demonstrating its effectiveness on both sparse and dense retrievers. Moreover, IterCQR exhibits superior performance in challenging settings such as generalization on unseen datasets and low-resource scenarios.

The mediator particle between the Standard Model sector and a hidden sector might have a long lifetime to show the observable displaced vertices in experiments. Considering a fermionic dark matter model in which the hidden sector is connected to the Standard Model by an additional Higgs doublet field, the mediator dark $Z$ boson may live long enough. We explore the possibility to observe the displaced vertices of the long-lived dark $Z$ boson at the CERN LHC and at the proposed SHiP experiment. We find that the ATLAS and CMS searches for the long-lived dark $Z$ boson can probe the mass range $7 < m_{Z'} < 150~{\rm MeV}$ with 150 fb$^{-1}$ integrated luminosity at the LHC run 3, and the SHiP experiment will probe $2 m_e < m_{Z'} < 15~{\rm MeV}$ range with $6 \times 10^{20}$ protons on target in total 15 years. The dark matter phenomenology is also discussed in the region where such a long-lived mediator is detectable., Comment: 11 pages, 4 figures

The problem of spurious programs is a longstanding challenge when training a semantic parser from weak supervision. To eliminate such programs that have wrong semantics but correct denotation, existing methods focus on exploiting similarities between examples based on domain-specific knowledge. In this paper, we propose a domain-agnostic filtering mechanism based on program execution results. Specifically, for each program obtained through the search process, we first construct a representation that captures the program's semantics as execution results under various inputs. Then, we run a majority vote on these representations to identify and filter out programs with significantly different semantics from the other programs. In particular, our method is orthogonal to the program search process so that it can easily augment any of the existing weakly supervised semantic parsing frameworks. Empirical evaluations on the Natural Language Visual Reasoning and WikiTableQuestions demonstrate that applying our method to the existing semantic parsers induces significantly improved performances., Comment: EMNLP 2023

The Mapper algorithm is a visualization technique in topological data analysis (TDA) that outputs a graph reflecting the structure of a given dataset. However, the Mapper algorithm requires tuning several parameters in order to generate a ``nice" Mapper graph. This paper focuses on selecting the cover parameter. We present an algorithm that optimizes the cover of a Mapper graph by splitting a cover repeatedly according to a statistical test for normality. Our algorithm is based on $G$-means clustering which searches for the optimal number of clusters in $k$-means by iteratively applying the Anderson-Darling test. Our splitting procedure employs a Gaussian mixture model to carefully choose the cover according to the distribution of the given data. Experiments for synthetic and real-world datasets demonstrate that our algorithm generates covers so that the Mapper graphs retain the essence of the datasets, while also running significantly fast.

Acquiring reliable microstructure datasets is a pivotal step toward the systematic design of materials with the aid of integrated computational materials engineering (ICME) approaches. However, obtaining three-dimensional (3D) microstructure datasets is often challenging due to high experimental costs or technical limitations, while acquiring two-dimensional (2D) micrographs is comparatively easier. To deal with this issue, this study proposes a novel framework for 2D-to-3D reconstruction of microstructures called Micro3Diff using diffusion-based generative models (DGMs). Specifically, this approach solely requires pre-trained DGMs for the generation of 2D samples, and dimensionality expansion (2D-to-3D) takes place only during the generation process (i.e., reverse diffusion process). The proposed framework incorporates a new concept referred to as multi-plane denoising diffusion, which transforms noisy samples (i.e., latent variables) from different planes into the data structure while maintaining spatial connectivity in 3D space. Furthermore, a harmonized sampling process is developed to address possible deviations from the reverse Markov chain of DGMs during the dimensionality expansion. Combined, we demonstrate the feasibility of Micro3Diff in reconstructing 3D samples with connected slices that maintain morphologically equivalence to the original 2D images. To validate the performance of Micro3Diff, various types of microstructures (synthetic and experimentally observed) are reconstructed, and the quality of the generated samples is assessed both qualitatively and quantitatively. The successful reconstruction outcomes inspire the potential utilization of Micro3Diff in upcoming ICME applications while achieving a breakthrough in comprehending and manipulating the latent space of DGMs.

We propose a novel approach for discovering a light dark photon in the forward experiments at the LHC, including the SND@LHC and the FASER experiments. Assuming the dark photon is lighter than twice the electron mass and feebly interacts with ordinary matter, it is long-lived enough to pass through 100 m of rock in front of the forward experiments and also through the detector targets. However, some portion of them could be converted into an electron-positron pair inside the detector through their interaction with the detector target, leaving an isolated electromagnetic shower as a clear new physics signature of the dark photon. With copiously produced dark photons from neutral pion decays in the forward region of the LHC, we expect to observe sizable events inside the detector. Our estimation shows that more than 10 signal events of the dark photon could be observed in the range of kinetic mixing parameter, $6.2\times10^{-5} \lesssim \epsilon \lesssim 2\times10^{-1}$ and $3\times10^{-5} \lesssim \epsilon \lesssim 2\times10^{-1}$ for dark photon mass $m_{A^\prime} \lesssim$ 1 MeV with integrated luminosities of 150 fb$^{-1}$ and 3 ab$^{-1}$, respectively., Comment: 11 pages, 3 figures, journal version

We study constraints on the hidden sector model mediated by an additional SU(2) Higgs doublet from the phenomenology of Higgs bosons. The hidden sector is assumed to contain a hidden U(1) gauge symmetry and the hidden U(1) gauge boson gets the mass by the electroweak symmetry breaking to be a dark Z boson. The Higgs sector of the model is similar to that of the two Higgs doublet model of type I except for the absence of the CP-odd scalar boson. Using the programs of HiggsBounds and HiggsSignals, we incorporate current experimental limits from LEP, Tevatron and LHC to examine the Higgs sector in our model and derive constraints on model parameters. We also discuss the implications of the model on the dark matter phenomenology., Comment: 18 pages, 7 figures

Gender bias is a significant issue in machine translation, leading to ongoing research efforts in developing bias mitigation techniques. However, most works focus on debiasing bilingual models without much consideration for multilingual systems. In this paper, we specifically target the gender bias issue of multilingual machine translation models for unambiguous cases where there is a single correct translation, and propose a bias mitigation method based on a novel approach. Specifically, we propose Gender-Aware Contrastive Learning, GACL, which encodes contextual gender information into the representations of non-explicit gender words. Our method is target language-agnostic and is applicable to pre-trained multilingual machine translation models via fine-tuning. Through multilingual evaluation, we show that our approach improves gender accuracy by a wide margin without hampering translation performance. We also observe that incorporated gender information transfers and benefits other target languages regarding gender accuracy. Finally, we demonstrate that our method is applicable and beneficial to models of various sizes., Comment: Accepted to EMNLP 2023 Main Conference

Let $\mathbb B^n$ be the unit ball in $\mathbb C^n$ and $\mathbb H^n$ be the homogeneous Siegel domain of the second kind which is biholomorphic to $\mathbb B^n$. We show that the K\"ahler potential of $\mathbb H^n$ is unique up to the automorphisms among K\"ahler potentials whose differentials have constant norms. As an application, we consider a domain $\Omega$ in $\mathbb C^n$, which is biholomorphic to $\mathbb B^n$. We show that if $\Omega$ is affine homogeneous, then it is affine equivalent to $\mathbb H^n$. Assume next that its canonical potential with respect to the K\"ahler--Einstein metric has a differential with a constant norm. If the biholomorphism between $\Omega$ and $\mathbb B^n$ is a restriction of a M\"obius transformation, then the map is affine equivalent to a Cayley transform., Comment: 22 pages

This paper puts forward an integrated microstructure design methodology that replaces the common existing design approaches: 1) reconstruction of microstructures, 2) analyzing and quantifying material properties, and 3) inverse design of materials using deep-learned generative and surrogate models. The long-standing issue of microstructure reconstruction is well addressed in this study using a new class of state-of-the-art generative model, the diffusion-based generative model (DGM). Moreover, the conditional formulation of DGM for guidance to the embedded desired material properties with a transformer-based attention mechanism enables the inverse design of multifunctional composites. A convolutional neural network (CNN)-based surrogate model is utilized to analyze the nonlinear material behavior to facilitate the prediction of material properties for building microstructure-property linkages. Combined, these generative and surrogate models enable large data processing and database construction that is often not affordable with resource-intensive finite element method (FEM)-based direct numerical simulation (DNS) and iterative reconstruction methods. An example case is presented to demonstrate the effectiveness of the proposed approach, which is designing mechanoluminescence (ML) particulate composites made of europium and dysprosium ions. The results show that the inversely-designed multiple ML microstructure candidates with the proposed generative and surrogate models meet the multiple design requirements (e.g., volume fraction, elastic constant, and light sensitivity). The evaluation of the generated samples' quality and the surrogate models' performance using appropriate metrics are also included. This assessment demonstrates that the proposed integrated methodology offers an end-to-end solution for practical material design applications.

Microstructure reconstruction has been an essential part of computational material engineering to reveal the relationship between microstructures and material properties. However, finding a general solution for microstructure characterization and reconstruction (MCR) tasks is still challenging, although there have been many attempts such as the descriptor-based MCR methods. To address this generality problem, the denoising diffusion models are first employed for the microstructure reconstruction task in this study. The applicability of the diffusion-based models is validated with several types of microstructures (e.g., polycrystalline alloy, carbonate, ceramics, copolymer, fiber composite, etc.) that have different morphological characteristics. The quality of the generated images is assessed with the quantitative evaluation metrics (FID score, precision, and recall) and the conventional statistical microstructure descriptors. Furthermore, the formulation of implicit probabilistic models (which yields non-Markovian diffusion processes) is adopted to accelerate the sampling process, thereby controlling the computational cost considering the practicability and reliability. The results show that the denoising diffusion models are well applicable to the reconstruction of various types of microstructures with different spatial distributions and morphological features. The diffusion-based approach provides a stable training process with simple implementation for generating visually similar and statistically equivalent microstructures. In these regards, the diffusion model has great potential to be used as a universal microstructure reconstruction method for handling complex microstructures for materials science.

We will show that a universal covering of a compact K\"ahler manifold with ample canonical bundle is the unit ball if it admits a global potential function of the K\"ahler-Einstein metric whose gradient length is a minimal constant. As an application, we will extend the Wong-Rosay theorem to a complex manifold without boundary.

In this paper, we deal with a strongly pseudoconvex almost CR manifold with a CR contraction. We will prove that the stable manifold of the CR contaction is CR equivalent to the Heisenberg group model., Comment: 16 pages

A graph can be regarded as an electrical network in which each edge is a resistor. This point of view relates combinatorial quantities, such as the number of spanning trees, to electrical ones such as effective resistance. The second and third authors have extended the combinatorics/electricity analogy to higher dimension and expressed the simplicial analogue of effective resistance as a ratio of weighted tree enumerators. In this paper, we first use that ratio to prove a new enumeration formula for color-shifted complexes, confirming a conjecture by Aalipour and the first author, and generalizing a result of Ehrenborg and van Willigenburg on Ferrers graphs. We then use the same technique to recover an enumeration formula for shifted complexes, first proved by Klivans and the first and fourth authors. In each case, we add facets one at a time, and give explicit expressions for simplicial effective resistances of added facets by constructing high-dimensional analogues of currents and voltages (respectively homological cycles and cohomological cocycles)., Comment: 27 pages, minor revisions from v1

We study a two-component dark matter model consisting of a Dirac fermion and a complex scalar charged under new U(1) gauge group in the hidden sector. The dark fermion plays the dominant component of dark matter which explains the measured DM relic density of the Universe. It has no direct coupling to ordinary standard model particles, thus evading strong constraints from the direct DM detection experiments. The dark fermion is self-interacting through the light dark gauge boson and it would be possible to address that this model can be a resolution to the small scale structure problem of the Universe. The light dark gauge boson, which interacts with the standard model sector, is also stable and composes the subdominant DM component. We investigate the model parameter space allowed by current experimental constraints and phenomenological bounds. We also discuss the sensitivity of future experiments such as SHiP, DUNE and ILC, for the obtained allowed parameter space., Comment: 13 pages, 1 figure, journal version