Search

Open-Vocabulary Video Instance Segmentation (VIS) is attracting increasing attention due to its ability to segment and track arbitrary objects. However, the recent Open-Vocabulary VIS attempts obtained unsatisfactory results, especially in terms of generalization ability of novel categories. We discover that the domain gap between the VLM features (e.g., CLIP) and the instance queries and the underutilization of temporal consistency are two central causes. To mitigate these issues, we design and train a novel Open-Vocabulary VIS baseline called OVFormer. OVFormer utilizes a lightweight module for unified embedding alignment between query embeddings and CLIP image embeddings to remedy the domain gap. Unlike previous image-based training methods, we conduct video-based model training and deploy a semi-online inference scheme to fully mine the temporal consistency in the video. Without bells and whistles, OVFormer achieves 21.9 mAP with a ResNet-50 backbone on LV-VIS, exceeding the previous state-of-the-art performance by 7.7. Extensive experiments on some Close-Vocabulary VIS datasets also demonstrate the strong zero-shot generalization ability of OVFormer (+ 7.6 mAP on YouTube-VIS 2019, + 3.9 mAP on OVIS). Code is available at https://github.com/fanghaook/OVFormer., Comment: ECCV 2024

Reset controllers have demonstrated their effectiveness in enhancing performance in precision motion systems. To further exploiting the potential of reset controllers, this study introduces a parallel-partial reset control structure. Frequency response analysis is effective for the design and fine-tuning of controllers in industries. However, conducting frequency response analysis for reset control systems poses challenges due to their nonlinearities. We develop frequency response analysis methods for both the open-loop and closed-loop parallel-partial reset systems. Simulation results validate the accuracy of the analysis methods, showcasing precision enhancements exceeding 100% compared to the traditional describing function method. Furthermore, we design a parallel-partial reset controller within the Proportional-Integral-Derivative (PID) control structure for a mass-spring-damper system. The frequency response analysis of the designed system indicates that, while maintaining the same bandwidth and phase margin of the first-order harmonics, the new system exhibits lower magnitudes of higher-order harmonics, compared to the traditional reset system. Moreover, simulation results demonstrate that the new system achieves lower overshoot and quicker settling time compared to both the traditional reset and linear systems., Comment: Submitted to The European Control Conference 2024

Fine-grained urban flow inference (FUFI) is a crucial transportation service aimed at improving traffic efficiency and safety. FUFI can infer fine-grained urban traffic flows based solely on observed coarse-grained data. However, most of existing methods focus on the influence of single-scale static geographic information on FUFI, neglecting the interactions and dynamic information between different-scale regions within the city. Different-scale geographical features can capture redundant information from the same spatial areas. In order to effectively learn multi-scale information across time and space, we propose an effective fine-grained urban flow inference model called UrbanMSR, which uses self-supervised contrastive learning to obtain dynamic multi-scale representations of neighborhood-level and city-level geographic information, and fuses multi-scale representations to improve fine-grained accuracy. The fusion of multi-scale representations enhances fine-grained. We validate the performance through extensive experiments on three real-world datasets. The resutls compared with state-of-the-art methods demonstrate the superiority of the proposed model.

Device-to-device (D2D) technology is one of the key research areas in 5G/6G networks, and full-duplex (FD) D2D will further enhance its spectral efficiency (SE). In recent years, deep learning approaches have shown remarkable performance in D2D resource allocation tasks. However, most schemes only model the channel state information (CSI) as an independent feature, neglecting the spatial relationships among multiple channels and users within the scenario. In this paper, we first design an objective function for FD D2D communication resource allocation, which aims to maximize the SE of D2D users while ensuring the minimal required SE of cellular users. Then, considering the complex CSI constituted by all the users in different channels as a three-dimensional vector, a centralized resource allocation model based on multi-dimensional spatial convolutional networks and attention mechanisms (SP-Conv-Att) is proposed. To alleviate the burden of base station, we develop two distributed models, Dist-Att and Dist-Att-Conv, to facilitate users to perform channel and power allocation locally, based on attention and multi-user convolutional networks respectively. Numerical results demonstrate that our models outperform traditional schemes and recent deep neural network models, significantly approximating the optimal solution computed by exhaustive algorithm with extremely low latency., Comment: 7 pages, 5 figures, 5 tables

Multimodal learning is susceptible to modality missing, which poses a major obstacle for its practical applications and, thus, invigorates increasing research interest. In this paper, we investigate two challenging problems: 1) when modality missing exists in the training data, how to exploit the incomplete samples while guaranteeing that they are properly supervised? 2) when the missing rates of different modalities vary, causing or exacerbating the imbalance among modalities, how to address the imbalance and ensure all modalities are well-trained? To tackle these two challenges, we first introduce the variational information bottleneck (VIB) method for the cross-modal representation learning of missing modalities, which capitalizes on the available modalities and the labels as supervision. Then, accounting for the imbalanced missing rates, we define relative advantage to quantify the advantage of each modality over others. Accordingly, a bi-level optimization problem is formulated to adaptively regulate the supervision of all modalities during training. As a whole, the proposed approach features \textbf{Re}lative a\textbf{d}vantage aware \textbf{C}ross-m\textbf{o}dal \textbf{r}epresentation l\textbf{e}arning (abbreviated as \textbf{RedCore}) for missing modalities with imbalanced missing rates. Extensive empirical results demonstrate that RedCore outperforms competing models in that it exhibits superior robustness against either large or imbalanced missing rates.

The ideal strength is crucial for predicting material behavior under extreme conditions, which can provide insights into material limits, guide design and engineer for enhanced performance and durability. In this work, we present a method within an allows for the estimation of tensile, shear, and indentation strengths in any crystallographic direction or plane. We have examined the strain-stress relationships of several well-known structures and compared our findings with previous work, demonstrating the effectiveness of our approach. Moreover, we performed extensive investigations into the indentation strength of hexagonal WC, \b{eta}-SiC, and MgA$l_2O_4$. The current study uncovers the modes of structural deformation and the underlying atomistic mechanisms. The insights gained from this study have significant implications for the further exploration and design of superhard materials., Comment: 15 pages,7 figures

Digital twin is a modern technology for many advanced applications. To construct a digital twin of a thermal system, it is required to make online estimations of unknown time-varying boundary conditions from sensor measured data, which needs to solve inverse heat transfer problems (IHTPs). However, a fast and accurate solution is challenging since the measured data is normally contaminated with noise and the traditional method to solve IHTP involves significant amount of calculations. Therefore, in this work, a rapid yet robust inversion algorithm called ANN-based extended Kalman smoothing algorithm is developed to realize the online prediction of desired parameter based on the measurements. The fast prediction is realized by replacing the conventional CFD-based state transfer models in extended Kalman smoothing algorithm with pre-trained ANN. Then, a two-dimensional internal convective heat transfer problem was employed as the case study to test the algorithm. The results have proved that the proposed algorithm is a computational-light and robust approach for solving IHTPs. The proposed algorithm can achieve estimation of unknown boundary conditions with a dimensionless average error of 0.0580 under noisy temperature measurement with a standard deviation of 10 K with a drastic reduction of computational cost compared to the conventional approach. Moreover, the effects of training data, location of sensor, future time step selection on the performance of prediction are investigated.

Speaker diarization is a task to label an audio or video recording with the identity of the speaker at each given time stamp. In this work, we propose a novel machine learning framework to conduct real-time multi-speaker diarization and recognition without prior registration and pretraining in a fully online and reinforcement learning setting. Our framework combines embedding extraction, clustering, and resegmentation into the same problem as an online decision-making problem. We discuss practical considerations and advanced techniques such as the offline reinforcement learning, semi-supervision, and domain adaptation to address the challenges of limited training data and out-of-distribution environments. Our approach considers speaker diarization as a fully online learning problem of the speaker recognition task, where the agent receives no pretraining from any training set before deployment, and learns to detect speaker identity on the fly through reward feedbacks. The paradigm of the reinforcement learning approach to speaker diarization presents an adaptive, lightweight, and generalizable system that is useful for multi-user teleconferences, where many people might come and go without extensive pre-registration ahead of time. Lastly, we provide a desktop application that uses our proposed approach as a proof of concept. To the best of our knowledge, this is the first approach to apply a reinforcement learning approach to the speaker diarization task.

In this paper, we study a bivariate distributionally robust optimization problem with mean-covariance ambiguity set and half-space support. Under a conventional type of objective function widely adopted in inventory management, option pricing, and portfolio selection, we obtain closed-form tight bounds of the inner problem in six different cases. Through a primal-dual approach, we identify the optimal distributions in each case. As an application in inventory control, we first derive the optimal order quantity and the corresponding worst-case distribution, extending the existing results in the literature. Moreover, we show that under the distributionally robust setting, a centralized inventory system does not necessarily reduce the optimal total inventory, which contradicts conventional wisdom. Furthermore, we identify two effects, a conventional pooling effect, and a novel shifting effect, the combination of which determines the benefit of incorporating the covariance information in the ambiguity set. Finally, we demonstrate through numerical experiments the importance of keeping the covariance information in the ambiguity set instead of compressing the information into one dimension.