Search

The goal of this paper is to improve the performance of pretrained Vision Transformer (ViT) models, particularly DINOv2, in image clustering task without requiring re-training or fine-tuning. As model size increases, high-norm artifacts anomaly appears in the patches of multi-head attention. We observe that this anomaly leads to reduced accuracy in zero-shot image clustering. These artifacts are characterized by disproportionately large values in the attention map compared to other patch tokens. To address these artifacts, we propose an approach called Inference-Time Attention Engineering (ITAE), which manipulates attention function during inference. Specifically, we identify the artifacts by investigating one of the Query-Key-Value (QKV) patches in the multi-head attention and attenuate their corresponding attention values inside the pretrained models. ITAE shows improved clustering accuracy on multiple datasets by exhibiting more expressive features in latent space. Our findings highlight the potential of ITAE as a practical solution for reducing artifacts in pretrained ViT models and improving model performance in clustering tasks without the need for re-training or fine-tuning., Comment: Accepted to ACCV 2024

We develop a large language model (LLM) based automatic speech recognition (ASR) system that can be contextualized by providing keywords as prior information in text prompts. We adopt decoder-only architecture and use our in-house LLM, PLaMo-100B, pre-trained from scratch using datasets dominated by Japanese and English texts as the decoder. We adopt a pre-trained Whisper encoder as an audio encoder, and the audio embeddings from the audio encoder are projected to the text embedding space by an adapter layer and concatenated with text embeddings converted from text prompts to form inputs to the decoder. By providing keywords as prior information in the text prompts, we can contextualize our LLM-based ASR system without modifying the model architecture to transcribe ambiguous words in the input audio accurately. Experimental results demonstrate that providing keywords to the decoder can significantly improve the recognition performance of rare and ambiguous words., Comment: 13 pages, 1 figure, and 7 tables

The Robinson-Trautman solution in the Einstein-Maxwell-$\Lambda$ system admits a shear-free and twist-free null geodesic congruence with a nonvanishing expansion. Restricting to the case where the Maxwell field is aligned, i.e., the spacetime is algebraically special, we provide an exhaustive classification of supersymmetric Robinson-Trautman spacetimes in the four-dimensional ${\cal N}=2$ gauged supergravity. The differential constraints that arise from the integrability conditions of the Killing spinor equation enable us to systematically reconstruct the metric. We derive the explicit form of the Killing spinor either by directly integrating the Killing spinor equation or by casting the solution into the canonical form of supersymmetric solutions given in hep-th/0307022. In any case, the supersymmetric Robinson-Trautman solution generically exhibits a naked singularity., Comment: v2: 28 pages, 1 figure, 1 table; refs added, appendix B removed, to appear in JHEP

In recent years, various subspace algorithms have been developed to handle large-scale optimization problems. Although existing subspace Newton methods require fewer iterations to converge in practice, the matrix operations and full gradient computation are bottlenecks when dealing with large-scale problems. %In this study, We propose a subspace quasi-Newton method that is restricted to a deterministic-subspace together with a gradient approximation based on random matrix theory. Our method does not require full gradients, let alone Hessian matrices. Yet, it achieves the same order of the worst-case iteration complexities in average for convex and nonconvex cases, compared to existing subspace methods. In numerical experiments, we confirm the superiority of our algorithm in terms of computation time.

Many image retrieval studies use metric learning to train an image encoder. However, metric learning cannot handle differences in users' preferences, and requires data to train an image encoder. To overcome these limitations, we revisit relevance feedback, a classic technique for interactive retrieval systems, and propose an interactive CLIP-based image retrieval system with relevance feedback. Our retrieval system first executes the retrieval, collects each user's unique preferences through binary feedback, and returns images the user prefers. Even when users have various preferences, our retrieval system learns each user's preference through the feedback and adapts to the preference. Moreover, our retrieval system leverages CLIP's zero-shot transferability and achieves high accuracy without training. We empirically show that our retrieval system competes well with state-of-the-art metric learning in category-based image retrieval, despite not training image encoders specifically for each dataset. Furthermore, we set up two additional experimental settings where users have various preferences: one-label-based image retrieval and conditioned image retrieval. In both cases, our retrieval system effectively adapts to each user's preferences, resulting in improved accuracy compared to image retrieval without feedback. Overall, our work highlights the potential benefits of integrating CLIP with classic relevance feedback techniques to enhance image retrieval., Comment: Accepted to ECCV 2024 Workshops: 2nd Workshop on Traditional Computer Vision in the Age of Deep Learning (TradiCV)

Nesterov's acceleration in continuous optimization can be understood in a novel way when Nesterov's accelerated gradient (NAG) method is considered as a linear multistep (LM) method for gradient flow. Although the NAG method for strongly convex functions (NAG-sc) has been fully discussed, the NAG method for $L$-smooth convex functions (NAG-c) has not. To fill this gap, we show that the existing NAG-c method can be interpreted as a variable step size LM (VLM) for the gradient flow. Surprisingly, the VLM allows linearly increasing step sizes, which explains the acceleration in the convex case. Here, we introduce a novel technique for analyzing the absolute stability of VLMs. Subsequently, we prove that NAG-c is optimal in a certain natural class of VLMs. Finally, we construct a new broader class of VLMs by optimizing the parameters in the VLM for ill-conditioned problems. According to numerical experiments, the proposed method outperforms the NAG-c method in ill-conditioned cases. These results imply that the numerical analysis perspective of the NAG is a promising working environment, and considering a broader class of VLMs could further reveal novel methods.

Zeroth-order optimization, which does not use derivative information, is one of the significant research areas in the field of mathematical optimization and machine learning. Although various studies have explored zeroth-order algorithms, one of the theoretical limitations is that oracle complexity depends on the dimension, i.e., on the number of variables, of the optimization problem. In this paper, to reduce the dependency of the dimension in oracle complexity, we propose a zeroth-order random subspace algorithm by combining a gradient-free algorithm (specifically, Gaussian randomized smoothing with central differences) with random projection. We derive the worst-case oracle complexity of our proposed method in non-smooth and convex settings; {\color{black} it is equivalent to standard results for full-dimensional non-smooth convex algorithms. Furthermore,} we prove that ours also has a local convergence rate independent of the original dimension under additional assumptions. In addition to the theoretical results, numerical experiments show that when an objective function has a specific structure, the proposed method can become experimentally more efficient due to random projection., Comment: 20 pages, 4 figures

The vacuum Robinson-Trautman solution admits a shear-free and twist-free null geodesic congruence with a nonvanishing expansion. We perform a comprehensive classification of solutions exhibiting this property in Einstein's gravity with a massless scalar field, assuming that the solution belongs at least to Petrov-type II and some of the components of Ricci tensor identically vanish. We find that these solutions can be grouped into three distinct classes: (I-a) a natural extension of the Robinson-Trautman family incorporating a scalar hair satisfying the time derivative of the Ricci flow equation, (I-b) a novel non-asymptotically flat solution characterized by two functions satisfying Perelman's pair of the Ricci flow equations, and (II) a dynamical solution possessing ${\rm SO}(3)$, ${\rm ISO}(2)$ or ${\rm SO}(1,2)$ symmetry. We provide a complete list of all explicit solutions falling into Petrov type D for classes (I-a) and (I-b). Moreover, leveraging the massless solution in class (I-a), we derive the neutral Robinson-Trautman solution to the ${\cal N}=2$ gauged supergravity with the prepotential $F(X) =-iX^0X^1$. By flipping the sign of the kinetic term of the scalar field, the Petrov-D class (I-a) solution leads to a time-dependent wormhole with an instantaneous spacetime singularity. Although the general solution is unavailable for class (II), we find a new dynamical solution with spherical symmetry from the AdS-Roberts solution via AdS/Ricci-flat correspondence., Comment: v2: 42 pages, 2 figures; clarifications amended, references added, to appear in CQG

We propose randomized subspace gradient methods for high-dimensional constrained optimization. While there have been similarly purposed studies on unconstrained optimization problems, there have been few on constrained optimization problems due to the difficulty of handling constraints. Our algorithms project gradient vectors onto a subspace that is a random projection of the subspace spanned by the gradients of active constraints. We determine the worst-case iteration complexity under linear and nonlinear settings and theoretically confirm that our algorithms can take a larger step size than their deterministic version. From the advantages of taking longer step and randomized subspace gradients, we show that our algorithms are especially efficient in view of time complexity when gradients cannot be obtained easily. Numerical experiments show that they tend to find better solutions because of the randomness of their subspace selection. Furthermore, they performs well in cases where gradients could not be obtained directly, and instead, gradients are obtained using directional derivatives., Comment: 38 pages, 4 figures

The C-metric in vacuum general relativity describes a pair of accelerated black holes supported by conical singularity. In this paper, we present a new family of exact solutions to the Einstein-phantom scalar system that describes accelerated wormholes in AdS. In the zero acceleration limit with a vanishing potential, the present solution recovers the asymptotically flat wormhole originally constructed by Ellis and Bronnikov. The scalar potential of the phantom field has an infinite number of critical points and is expressed in terms of the superpotential, which is obtained by suitable analytic continuation of one parameter family of the ${\cal N}=2$ gauged supergravity. As one traverses two asymptotic regions connected by throat, the scalar field evolves from AdS, corresponding to the origin of the potential, towards the neighboring AdS local minimum of the potential. We find that the flipping transformation, which interchanges the role of ``radial'' and "angular'' coordinates at the expense of double Wick rotation, is an immediate cause for the existence of two branches of static AdS wormholes discovered previously. Contrary to the ordinary C-metric, the conical singularity along the symmetry axis can be completely resolved, when the (super)potential is periodic or zero. We explore the global causal structure in detail., Comment: 17 pages, 5 figures; v2 references added, accepted version in PRD