Search

Recent research into buffet in the transonic flow regime has been focused on a limited number of proprietary airfoil geometries and has mainly considered parametric variations in Mach number and angle of attack. In contrast, relatively little is known about the sensitivity of buffet frequencies and amplitudes to geometric properties of airfoils. In the present contribution, an airfoil geometry construction method based on a small number of parameters is developed. The resulting airfoils and computational grids are high-order continuous everywhere except at the trailing edge corners. The effects of four key geometric parameters, defined by local extrema of coordinates on the airfoil at the design condition and denoted as 'crest' points, are studied using large-eddy simulation, considering both free-transitional and tripped boundary layers. For both states of the boundary layer, buffet amplitude and frequency are found to be highly sensitive to the axial and vertical position of the suction-side crest point, while the vertical crest position on the pressure side affects the mean lift. The present work confirms that buffet can appear for free-transitional (laminar buffet) and tripped conditions (turbulent buffet) with similar sensitivities. For test cases near onset conditions intermediate-frequency phenomena have been observed, which were linked to unsteadiness of separation bubbles. Frequencies scaling based on mean-flow properties of the separation bubble were shown to be in good agreement with previous findings on different airfoil geometries in Zauner et al. (Flow Turb. & Comb., Vol. 110, 2023, pp. 1023-1057). Airfoil geometries are provided as open source: https://doi.org/10.5281/zenodo.12204411, Comment: AIAA AVIATION FORUM AND ASCEND 2024, Session: Aerodynamic Design I

Tactile perception is vital, especially when distinguishing visually similar objects. We propose an approach to incorporate tactile data into a Vision-Language Model (VLM) for visuo-tactile zero-shot object recognition. Our approach leverages the zero-shot capability of VLMs to infer tactile properties from the names of tactilely similar objects. The proposed method translates tactile data into a textual description solely by annotating object names for each tactile sequence during training, making it adaptable to various contexts with low training costs. The proposed method was evaluated on the FoodReplica and Cube datasets, demonstrating its effectiveness in recognizing objects that are difficult to distinguish by vision alone., Comment: 9 pages, 9 figures, accepted to IROS2024, project page: https://omron-sinicx.github.io/visuo-tactile-recognition/

Procedural video understanding is gaining attention in the vision and language community. Deep learning-based video analysis requires extensive data. Consequently, existing works often use web videos as training resources, making it challenging to query instructional contents from raw video observations. To address this issue, we propose a new dataset, COM Kitchens. The dataset consists of unedited overhead-view videos captured by smartphones, in which participants performed food preparation based on given recipes. Fixed-viewpoint video datasets often lack environmental diversity due to high camera setup costs. We used modern wide-angle smartphone lenses to cover cooking counters from sink to cooktop in an overhead view, capturing activity without in-person assistance. With this setup, we collected a diverse dataset by distributing smartphones to participants. With this dataset, we propose the novel video-to-text retrieval task Online Recipe Retrieval (OnRR) and new video captioning domain Dense Video Captioning on unedited Overhead-View videos (DVC-OV). Our experiments verified the capabilities and limitations of current web-video-based SOTA methods in handling these tasks., Comment: ECCV2024 accepted

Visual question answering aims to provide responses to natural language questions given visual input. Recently, visual programmatic models (VPMs), which generate executable programs to answer questions through large language models (LLMs), have attracted research interest. However, they often require long input prompts to provide the LLM with sufficient API usage details to generate relevant code. To address this limitation, we propose AdaCoder, an adaptive prompt compression framework for VPMs. AdaCoder operates in two phases: a compression phase and an inference phase. In the compression phase, given a preprompt that describes all API definitions in the Python language with example snippets of code, a set of compressed preprompts is generated, each depending on a specific question type. In the inference phase, given an input question, AdaCoder predicts the question type and chooses the appropriate corresponding compressed preprompt to generate code to answer the question. Notably, AdaCoder employs a single frozen LLM and pre-defined prompts, negating the necessity of additional training and maintaining adaptability across different powerful black-box LLMs such as GPT and Claude. In experiments, we apply AdaCoder to ViperGPT and demonstrate that it reduces token length by 71.1%, while maintaining or even improving the performance of visual question answering.

To reduce the time-to-market of future aircraft, it is crucial to predict the flight envelope accurately before building prototypes for flight tests. The High-Lift Prediction Workshop (HLPW) series aims to assess the numerical prediction capability of current CFD technology considering NASA's high-lift version of the Common Research Model (CRM-HL). The present work contributes to these collaborative efforts, quantifying sensitivities for RANS-based steady, unsteady, and hybrid RANS/LES scale-resolving approaches. Uncertainties associated with the choice of turbulence model, initialization strategies, grid resolution, and iterative convergence at free-air conditions are covered. Near stall, a large spread of RANS results was observed for different turbulence models and initialization strategies, while iterative convergence appeared less crucial for the present simulations. Steady and unsteady RANS simulations were unable to predict the correct flow physics near CLmax, even for large grids. Delayed Detached Eddy Simulations (DDES), however, showed good accuracy compared with wind-tunnel experiments and predicted CLmax with an error of around 5%. Compared to steady RANS, the computational cost of DDES was a factor of ten higher. Lessons learned and potential best-practice strategies are shared to aid future studies. While warm-started RANS simulations using SA models are recommended at lower angles of attack, scale-resolving methods are required near stall.

Turbulent transonic buffet is an aerodynamic instability causing periodic oscillations of lift/drag in aerospace applications. Involving complex coupling between inviscid and viscous effects, buffet is characterised by shock-wave oscillations and flow separation/reattachment. Previous studies have identified both 2D chordwise shock-oscillation and 3D buffet/stall-cell modes. While the 2D instability has been studied extensively, investigations of 3D buffet have been limited to only low-fidelity simulations or experiments. Due to computational costs, almost all high-fidelity studies to date have been limited to narrow span-widths around 5% of aerofoil chord length (aspect ratio, $AR = 0.05$), which is insufficiently wide to observe large-scale three-dimensionality. In this work, high-fidelity simulations are performed up to $AR=3$, on infinite unswept NASA-CRM wing profiles at $Re=5\times 10^{5}$. At $AR \geq 1$, intermittent 3D separation bubbles are observed at buffet conditions. While previous RANS/stability-based studies predict simultaneous onset of 2D- and 3D-buffet, a case with buffet that remains essentially-2D despite span-widths up to $AR=2$ is identified here. Strongest three-dimensionality was observed near the low-lift phases of the buffet cycle at maximum flow separation, reverting to essentially-2D behaviour during high-lift phases. Buffet was found to become three-dimensional when extensive mean flow separation was present. At $AR \geq 2$, multiple 3D separation bubbles form, in a wavelength range of $\lambda=\left[1c-1.5c\right]$. SPOD and cross-correlations were applied to analyse the spatio/temporal structure of 3D buffet-cells. In addition to the 2D chordwise shock-oscillation mode (Strouhal number $St \approx 0.07-0.1$), 3D modal structures were found in the shocked region of the flow at $St \approx 0.002-0.004$.

Transonic buffet is an instability characterized by shock-oscillations and separated boundary-layers. High-fidelity simulations have typically been limited to narrow domains to be computationally feasible, overly constraining the flow and introducing modelling errors. Depending on the boundary-layer state upstream of the interaction, different buffet features are observed. High-fidelity simulations were performed on the periodic NASA-CRM wing at moderate Reynolds number to assess sensitivity of the two-dimensional transonic buffet to boundary-layer state and domain width. Simulations were cross-validated against RANS/URANS and global stability analysis and excellent agreement was found near the onset. By varying the boundary-layer tripping amplitude, laminar, transitional, and turbulent buffet interactions were obtained. All cases consisted of a single shock and low-frequency oscillations $(St \approx 0.07)$. The transitional interaction also exhibited reduced shock movement, a $15\%$ increase in $\overline{C_L}$, and energy content at higher frequencies $(St \approx 1.3)$. Span-wise domain studies showed sensitivity at the shock location and near the trailing edge. We conclude that the span-width must be greater than the trailing-edge boundary-layer thickness to obtain span-independent solutions. For largely separated cases, the sensitivity to span-width increased and variations across the span were observed. This was found to be associated to a loss of two-dimensionality of the flow.

Large language models (LLMs) are accelerating the development of language-guided robot planners. Meanwhile, symbolic planners offer the advantage of interpretability. This paper proposes a new task that bridges these two trends, namely, multimodal planning problem specification. The aim is to generate a problem description (PD), a machine-readable file used by the planners to find a plan. By generating PDs from language instruction and scene observation, we can drive symbolic planners in a language-guided framework. We propose a Vision-Language Interpreter (ViLaIn), a new framework that generates PDs using state-of-the-art LLM and vision-language models. ViLaIn can refine generated PDs via error message feedback from the symbolic planner. Our aim is to answer the question: How accurately can ViLaIn and the symbolic planner generate valid robot plans? To evaluate ViLaIn, we introduce a novel dataset called the problem description generation (ProDG) dataset. The framework is evaluated with four new evaluation metrics. Experimental results show that ViLaIn can generate syntactically correct problems with more than 99\% accuracy and valid plans with more than 58\% accuracy. Our code and dataset are available at https://github.com/omron-sinicx/ViLaIn., Comment: ICRA 2024

Matching, a task to optimally assign limited resources under constraints, is a fundamental technology for society. The task potentially has various objectives, conditions, and constraints; however, the efficient neural network architecture for matching is underexplored. This paper proposes a novel graph neural network (GNN), \textit{WeaveNet}, designed for bipartite graphs. Since a bipartite graph is generally dense, general GNN architectures lose node-wise information by over-smoothing when deeply stacked. Such a phenomenon is undesirable for solving matching problems. WeaveNet avoids it by preserving edge-wise information while passing messages densely to reach a better solution. To evaluate the model, we approximated one of the \textit{strongly NP-hard} problems, \textit{fair stable matching}. Despite its inherent difficulties and the network's general purpose design, our model reached a comparative performance with state-of-the-art algorithms specially designed for stable matching for small numbers of agents.

In recent years large model trained on huge amount of cross-modality data, which is usually be termed as foundation model, achieves conspicuous accomplishment in many fields, such as image recognition and generation. Though achieving great success in their original application case, it is still unclear whether those foundation models can be applied to other different downstream tasks. In this paper, we conduct a short survey on the current methods for discriminative dense recognition tasks, which are built on the pretrained foundation model. And we also provide some preliminary experimental analysis of an existing open-vocabulary segmentation method based on Stable Diffusion, which indicates the current way of deploying diffusion model for segmentation is not optimal. This aims to provide insights for future research on adopting foundation model for downstream task., Comment: This is a short report on the current usage of foundation model (mainly pretrained diffusion model) for downstream dense recognition task (e.g., open vocabulary segmentation). We hope this short report could give an insight to the future research

To transfer the knowledge learned from a labeled source domain to an unlabeled target domain, many studies have worked on universal domain adaptation (UniDA), where there is no constraint on the label sets of the source domain and target domain. However, the existing UniDA methods rely on source samples with correct annotations. Due to the limited resources in the real world, it is difficult to obtain a large amount of perfectly clean labeled data in a source domain in some applications. As a result, we propose a novel realistic scenario named Noisy UniDA, in which classifiers are trained using noisy labeled data from the source domain as well as unlabeled domain data from the target domain that has an uncertain class distribution. A multi-head convolutional neural network framework is proposed in this paper to address all of the challenges faced in the Noisy UniDA at once. Our network comprises a single common feature generator and multiple classifiers with various decision bounds. We can detect noisy samples in the source domain, identify unknown classes in the target domain, and align the distribution of the source and target domains by optimizing the divergence between the outputs of the various classifiers. The proposed method outperformed the existing methods in most of the settings after a thorough analysis of the various domain adaption scenarios. The source code is available at \url{https://github.com/YU1ut/Divergence-Optimization}.

This paper tackles recipe generation from unsegmented cooking videos, a task that requires agents to (1) extract key events in completing the dish and (2) generate sentences for the extracted events. Our task is similar to dense video captioning (DVC), which aims at detecting events thoroughly and generating sentences for them. However, unlike DVC, in recipe generation, recipe story awareness is crucial, and a model should extract an appropriate number of events in the correct order and generate accurate sentences based on them. We analyze the output of the DVC model and confirm that although (1) several events are adoptable as a recipe story, (2) the generated sentences for such events are not grounded in the visual content. Based on this, we set our goal to obtain correct recipes by selecting oracle events from the output events and re-generating sentences for them. To achieve this, we propose a transformer-based multimodal recurrent approach of training an event selector and sentence generator for selecting oracle events from the DVC's events and generating sentences for them. In addition, we extend the model by including ingredients to generate more accurate recipes. The experimental results show that the proposed method outperforms state-of-the-art DVC models. We also confirm that, by modeling the recipe in a story-aware manner, the proposed model outputs the appropriate number of events in the correct order., Comment: Accepted at ACM TOMM; ACM Transactions on Multimedia Computing, Communications, and Applications

We present a new multimodal dataset called Visual Recipe Flow, which enables us to learn each cooking action result in a recipe text. The dataset consists of object state changes and the workflow of the recipe text. The state change is represented as an image pair, while the workflow is represented as a recipe flow graph (r-FG). The image pairs are grounded in the r-FG, which provides the cross-modal relation. With our dataset, one can try a range of applications, from multimodal commonsense reasoning and procedural text generation., Comment: COLING 2022

We present a novel differential matching algorithm for 3D point cloud registration. Instead of only optimizing the feature extractor for a matching algorithm, we propose a learning-based matching module optimized to the jointly-trained feature extractor. We focused on edge-wise feature-forwarding architectures, which are memory-consuming but can avoid the over-smoothing effect that GNNs suffer. We improve its memory efficiency to scale it for point cloud registration while investigating the best way of connecting it to the feature extractor. Experimental results show our matching module's significant impact on performance improvement in rigid/non-rigid and whole/partial point cloud registration datasets with multiple contemporary feature extractors. For example, our module boosted the current SOTA method, RoITr, by +5.4%, and +7.2% in the NFMR metric and +6.1% and +8.5% in the IR metric on the 4DMatch and 4DLoMatch datasets, respectively.

Hand segmentation is a crucial task in first-person vision. Since first-person images exhibit strong bias in appearance among different environments, adapting a pre-trained segmentation model to a new domain is required in hand segmentation. Here, we focus on appearance gaps for hand regions and backgrounds separately. We propose (i) foreground-aware image stylization and (ii) consensus pseudo-labeling for domain adaptation of hand segmentation. We stylize source images independently for the foreground and background using target images as style. To resolve the domain shift that the stylization has not addressed, we apply careful pseudo-labeling by taking a consensus between the models trained on the source and stylized source images. We validated our method on domain adaptation of hand segmentation from real and simulation images. Our method achieved state-of-the-art performance in both settings. We also demonstrated promising results in challenging multi-target domain adaptation and domain generalization settings. Code is available at https://github.com/ut-vision/FgSty-CPL., Comment: Accepted to IEEE Access 2021

Unsupervised image captioning is a challenging task that aims at generating captions without the supervision of image-sentence pairs, but only with images and sentences drawn from different sources and object labels detected from the images. In previous work, pseudo-captions, i.e., sentences that contain the detected object labels, were assigned to a given image. The focus of the previous work was on the alignment of input images and pseudo-captions at the sentence level. However, pseudo-captions contain many words that are irrelevant to a given image. In this work, we investigate the effect of removing mismatched words from image-sentence alignment to determine how they make this task difficult. We propose a simple gating mechanism that is trained to align image features with only the most reliable words in pseudo-captions: the detected object labels. The experimental results show that our proposed method outperforms the previous methods without introducing complex sentence-level learning objectives. Combined with the sentence-level alignment method of previous work, our method further improves its performance. These results confirm the importance of careful alignment in word-level details., Comment: EACL 2021 (11 pages, 3 figures; added references)

Universal domain adaptation (UniDA) has been proposed to transfer knowledge learned from a label-rich source domain to a label-scarce target domain without any constraints on the label sets. In practice, however, it is difficult to obtain a large amount of perfectly clean labeled data in a source domain with limited resources. Existing UniDA methods rely on source samples with correct annotations, which greatly limits their application in the real world. Hence, we consider a new realistic setting called Noisy UniDA, in which classifiers are trained with noisy labeled data from the source domain and unlabeled data with an unknown class distribution from the target domain. This paper introduces a two-head convolutional neural network framework to solve all problems simultaneously. Our network consists of one common feature generator and two classifiers with different decision boundaries. By optimizing the divergence between the two classifiers' outputs, we can detect noisy source samples, find "unknown" classes in the target domain, and align the distribution of the source and target domains. In an extensive evaluation of different domain adaptation settings, the proposed method outperformed existing methods by a large margin in most settings., Comment: CVPR 2021

This paper proposes a novel approach for unsupervised domain adaptation (UDA) with target shift. Target shift is a problem of mismatch in label distribution between source and target domains. Typically it appears as class-imbalance in target domain. In practice, this is an important problem in UDA; as we do not know labels in target domain datasets, we do not know whether or not its distribution is identical to that in the source domain dataset. Many traditional approaches achieve UDA with distribution matching by minimizing mean maximum discrepancy or adversarial training; however these approaches implicitly assume a coincidence in the distributions and do not work under situations with target shift. Some recent UDA approaches focus on class boundary and some of them are robust to target shift, but they are only applicable to classification and not to regression. To overcome the target shift problem in UDA, the proposed method, partially shared variational autoencoders (PS-VAEs), uses pair-wise feature alignment instead of feature distribution matching. PS-VAEs inter-convert domain of each sample by a CycleGAN-based architecture while preserving its label-related content. To evaluate the performance of PS-VAEs, we carried out two experiments: UDA with class-unbalanced digits datasets (classification), and UDA from synthesized data to real observation in human-pose-estimation (regression). The proposed method presented its robustness against the class-imbalance in the classification task, and outperformed the other methods in the regression task with a large margin.

This work addresses a new problem that learns generative adversarial networks (GANs) from multiple data collections that are each i) owned separately by different clients and ii) drawn from a non-identical distribution that comprises different classes. Given such non-iid data as input, we aim to learn a distribution involving all the classes input data can belong to, while keeping the data decentralized in each client storage. Our key contribution to this end is a new decentralized approach for learning GANs from non-iid data called Forgiver-First Update (F2U), which a) asks clients to train an individual discriminator with their own data and b) updates a generator to fool the most `forgiving' discriminators who deem generated samples as the most real. Our theoretical analysis proves that this updating strategy allows the decentralized GAN to achieve a generator's distribution with all the input classes as its global optimum based on f-divergence minimization. Moreover, we propose a relaxed version of F2U called Forgiver-First Aggregation (F2A) that performs well in practice, which adaptively aggregates the discriminators while emphasizing forgiving ones. Our empirical evaluations with image generation tasks demonstrated the effectiveness of our approach over state-of-the-art decentralized learning methods.

Images captured in participating media such as murky water, fog, or smoke are degraded by scattered light. Thus, the use of traditional three-dimensional (3D) reconstruction techniques in such environments is difficult. In this paper, we propose a photometric stereo method for participating media. The proposed method differs from previous studies with respect to modeling shape-dependent forward scatter. In the proposed model, forward scatter is described as an analytical form using lookup tables and is represented by spatially-variant kernels. We also propose an approximation of a large-scale dense matrix as a sparse matrix, which enables the removal of forward scatter. Experiments with real and synthesized data demonstrate that the proposed method improves 3D reconstruction in participating media., Comment: 9 pages, accepted to CVPR 2018

Outlier detection and cluster number estimation is an important issue for clustering real data. This paper focuses on spectral clustering, a time-tested clustering method, and reveals its important properties related to outliers. The highlights of this paper are the following two mathematical observations: first, spectral clustering's intrinsic property of an outlier cluster formation, and second, the singularity of an outlier cluster with a valid cluster number. Based on these observations, we designed a function that evaluates clustering and outlier detection results. In experiments, we prepared two scenarios, face clustering in photo album and person re-identification in a camera network. We confirmed that the proposed method detects outliers and estimates the number of clusters properly in both problems. Our method outperforms state-of-the-art methods in both the 128-dimensional sparse space for face clustering and the 4,096-dimensional non-sparse space for person re-identification., Comment: 10 pages, 2 figures, 2 tables