Search

Contextual information is vital for accurate trajectory prediction. For instance, the intricate flying behavior of migratory birds hinges on their analysis of environmental cues such as wind direction and air pressure. However, the diverse and dynamic nature of contextual information renders it an arduous task for AI models to comprehend its impact on trajectories and consequently predict them accurately. To address this issue, we propose a ``manager-worker'' framework to unleash the full potential of contextual information and construct CATP model, an implementation of the framework for Context-Aware Trajectory Prediction. The framework comprises a manager model, several worker models, and a tailored training mechanism inspired by competition symbiosis in nature. Taking CATP as an example, each worker needs to compete against others for training data and develop an advantage in predicting specific moving patterns. The manager learns the workers' performance in different contexts and selects the best one in the given context to predict trajectories, enabling CATP as a whole to operate in a symbiotic manner. We conducted two comparative experiments and an ablation study to quantitatively evaluate the proposed framework and CATP model. The results showed that CATP could outperform SOTA models, and the framework could be generalized to different context-aware tasks.

Reinforcement learning (RL) has proven highly effective in addressing complex decision-making and control tasks. However, in most traditional RL algorithms, the policy is typically parameterized as a diagonal Gaussian distribution with learned mean and variance, which constrains their capability to acquire complex policies. In response to this problem, we propose an online RL algorithm termed diffusion actor-critic with entropy regulator (DACER). This algorithm conceptualizes the reverse process of the diffusion model as a novel policy function and leverages the capability of the diffusion model to fit multimodal distributions, thereby enhancing the representational capacity of the policy. Since the distribution of the diffusion policy lacks an analytical expression, its entropy cannot be determined analytically. To mitigate this, we propose a method to estimate the entropy of the diffusion policy utilizing Gaussian mixture model. Building on the estimated entropy, we can learn a parameter $\alpha$ that modulates the degree of exploration and exploitation. Parameter $\alpha$ will be employed to adaptively regulate the variance of the added noise, which is applied to the action output by the diffusion model. Experimental trials on MuJoCo benchmarks and a multimodal task demonstrate that the DACER algorithm achieves state-of-the-art (SOTA) performance in most MuJoCo control tasks while exhibiting a stronger representational capacity of the diffusion policy.

Interface engineered ferroelectricity in van der Waals heterostructures is of broad interest both fundamentally and technologically for the applications in neuromorphic computing and so on. In particular, the moir\'e ferroelectricity in graphene/hexagonal boron nitride (hBN) heterostructures driven by charge ordering instead of traditional lattice displacement has drawn considerable attention because of its fascinating properties and promising high-frequency programmable electrical polarization switching. Yet, the underlying mechanism of the electronic ferroelectricity is still under debate. On the other hand, combining the interface engineered ferroelectricity and strong correlations in moir\'e heterostructures could enable the realization of novel quantum states such as ferroelectric superconductivity and multiferroicity. Here we study the electronic transport properties of twisted double bilayer graphene (TDBLG), aligned with one of the neighbouring hBN. We observe a strong gating hysteresis and ferroelectric-like behaviour, as well as the electronic ratchet effect. We find that the top gate is anomalously screened. On the contrary, the back gate is anomalously doubly efficient in injecting charges into graphene, that is, the effective back gate capacitance is two times larger than its geometry capacitance. This unexpected gate-tunable capacitance causes a dramatic change of electric fields between forward and backward scans. The asymmetric gating behaviours and anomalous change in capacitance could be explained with a simple model involved with a spontaneous electric polarization between top hBN and graphene. Our work provides more insights into the mysterious ferroelectricity in graphene/hBN moir\'e heterostructures and paves the way to the understanding of the underlying mechanism., Comment: 20 pages, 13 figures

In the burgeoning field of large language models (LLMs), the assessment of fundamental knowledge remains a critical challenge, particularly for models tailored to Chinese language and culture. This paper introduces FoundaBench, a pioneering benchmark designed to rigorously evaluate the fundamental knowledge capabilities of Chinese LLMs. FoundaBench encompasses a diverse array of 3354 multiple-choice questions across common sense and K-12 educational subjects, meticulously curated to reflect the breadth and depth of everyday and academic knowledge. We present an extensive evaluation of 12 state-of-the-art LLMs using FoundaBench, employing both traditional assessment methods and our CircularEval protocol to mitigate potential biases in model responses. Our results highlight the superior performance of models pre-trained on Chinese corpora, and reveal a significant disparity between models' reasoning and memory recall capabilities. The insights gleaned from FoundaBench evaluations set a new standard for understanding the fundamental knowledge of LLMs, providing a robust framework for future advancements in the field.

This paper discusses the results of the third edition of the Monocular Depth Estimation Challenge (MDEC). The challenge focuses on zero-shot generalization to the challenging SYNS-Patches dataset, featuring complex scenes in natural and indoor settings. As with the previous edition, methods can use any form of supervision, i.e. supervised or self-supervised. The challenge received a total of 19 submissions outperforming the baseline on the test set: 10 among them submitted a report describing their approach, highlighting a diffused use of foundational models such as Depth Anything at the core of their method. The challenge winners drastically improved 3D F-Score performance, from 17.51% to 23.72%., Comment: To appear in CVPRW2024

Matching cost aggregation plays a fundamental role in learning-based multi-view stereo networks. However, directly aggregating adjacent costs can lead to suboptimal results due to local geometric inconsistency. Related methods either seek selective aggregation or improve aggregated depth in the 2D space, both are unable to handle geometric inconsistency in the cost volume effectively. In this paper, we propose GoMVS to aggregate geometrically consistent costs, yielding better utilization of adjacent geometries. More specifically, we correspond and propagate adjacent costs to the reference pixel by leveraging the local geometric smoothness in conjunction with surface normals. We achieve this by the geometric consistent propagation (GCP) module. It computes the correspondence from the adjacent depth hypothesis space to the reference depth space using surface normals, then uses the correspondence to propagate adjacent costs to the reference geometry, followed by a convolution for aggregation. Our method achieves new state-of-the-art performance on DTU, Tanks & Temple, and ETH3D datasets. Notably, our method ranks 1st on the Tanks & Temple Advanced benchmark., Comment: CVPR 2024. Project page: https://wuuu3511.github.io/gomvs/ Code: https://github.com/Wuuu3511/GoMVS

In this study, we have derived a thermodynamically consistent phase-field model for two-phase flows with thermocapillary effects. This model accommodates variations in physical properties such as density, viscosity, heat capacity, and thermal conductivity between the two components. The model equations encompass a Cahn-Hilliard equation with the volume fraction as the phase variable, a Navier-Stokes equation, and a heat equation, and meanwhile maintains mass conservation, energy conservation, and entropy increase simultaneously. Given the highly coupled and nonlinear nature of the model equations, we developed a semi-decoupled, mass-preserving, and entropy-stable time-discrete numerical method. We conducted several numerical tests to validate both our model and numerical method. Additionally, we have investigated the merging process of two bubbles under non-isothermal conditions and compared the results with those under isothermal conditions. Our findings reveal that temperature gradients influence bubble morphology and lead to earlier merging. Moreover, we have observed that the merging of bubbles slows down with increasing heat Peclect number PeT when the initial temperature field increases linearly along the channel, while bubbles merge faster with heat Peclect number PeT when the initial temperature field decreases linearly along the channel.

The generic large Vision-Language Models (VLMs) is rapidly developing, but still perform poorly in Remote Sensing (RS) domain, which is due to the unique and specialized nature of RS imagery and the comparatively limited spatial perception of current VLMs. Existing Remote Sensing specific Vision Language Models (RSVLMs) still have considerable potential for improvement, primarily owing to the lack of large-scale, high-quality RS vision-language datasets. We constructed HqDC-1.4M, the large scale High quality and Detailed Captions for RS images, containing 1.4 million image-caption pairs, which not only enhance the RSVLM's understanding of RS images but also significantly improve the model's spatial perception abilities, such as localization and counting, thereby increasing the helpfulness of the RSVLM. Moreover, to address the inevitable "hallucination" problem in RSVLM, we developed RSSA, the first dataset aimed at enhancing the Self-Awareness capability of RSVLMs. By incorporating a variety of unanswerable questions into typical RS visual question-answering tasks, RSSA effectively improves the truthfulness and reduces the hallucinations of the model's outputs, thereby enhancing the honesty of the RSVLM. Based on these datasets, we proposed the H2RSVLM, the Helpful and Honest Remote Sensing Vision Language Model. H2RSVLM has achieved outstanding performance on multiple RS public datasets and is capable of recognizing and refusing to answer the unanswerable questions, effectively mitigating the incorrect generations. We will release the code, data and model weights at https://github.com/opendatalab/H2RSVLM ., Comment: Equal contribution: Chao Pang, Jiang Wu; Corresponding author: Gui-Song Xia, Conghui He

Functional integral representations for solutions of the motion equations for wall-bounded incompressible viscous flows, expressed (implicitly) in terms of distributions of solutions to stochastic differential equations of McKean-Vlasov type, are established by using a perturbation technique. These representations are used to obtain exact random vortex dynamics for wall-bounded viscous flows. Numerical schemes therefore are proposed and the convergence of the numerical schemes for random vortex dynamics with an additional force term is established. Several numerical experiments are carried out for demonstrating the motion of a viscous flow within a thin layer next to the fluid boundary., Comment: 45 pages, 12 figures

We introduce CHARM, the first benchmark for comprehensively and in-depth evaluating the commonsense reasoning ability of large language models (LLMs) in Chinese, which covers both globally known and Chinese-specific commonsense. We evaluated 7 English and 12 Chinese-oriented LLMs on CHARM, employing 5 representative prompt strategies for improving LLMs' reasoning ability, such as Chain-of-Thought. Our findings indicate that the LLM's language orientation and the task's domain influence the effectiveness of the prompt strategy, which enriches previous research findings. We built closely-interconnected reasoning and memorization tasks, and found that some LLMs struggle with memorizing Chinese commonsense, affecting their reasoning ability, while others show differences in reasoning despite similar memorization performance. We also evaluated the LLMs' memorization-independent reasoning abilities and analyzed the typical errors. Our study precisely identified the LLMs' strengths and weaknesses, providing the clear direction for optimization. It can also serve as a reference for studies in other fields. We will release CHARM at https://github.com/opendatalab/CHARM ., Comment: Equal contribution: Jiaxing Sun, Weiquan Huang, Jiang Wu; Corresponding author: Conghui He

Large Language Models (LLMs) are a class of generative AI models built using the Transformer network, capable of leveraging vast datasets to identify, summarize, translate, predict, and generate language. LLMs promise to revolutionize society, yet training these foundational models poses immense challenges. Semantic vector search within large language models is a potent technique that can significantly enhance search result accuracy and relevance. Unlike traditional keyword-based search methods, semantic search utilizes the meaning and context of words to grasp the intent behind queries and deliver more precise outcomes. Elasticsearch emerges as one of the most popular tools for implementing semantic search an exceptionally scalable and robust search engine designed for indexing and searching extensive datasets. In this article, we delve into the fundamentals of semantic search and explore how to harness Elasticsearch and Transformer models to bolster large language model processing paradigms. We gain a comprehensive understanding of semantic search principles and acquire practical skills for implementing semantic search in real-world model application scenarios.

With the large language model showing human-like logical reasoning and understanding ability, whether agents based on the large language model can simulate the interaction behavior of real users, so as to build a reliable virtual recommendation A/B test scene to help the application of recommendation research is an urgent, important and economic value problem. The combination of interaction design and machine learning can provide a more efficient and personalized user experience for products and services. This personalized service can meet the specific needs of users and improve user satisfaction and loyalty. Second, the interactive system can understand the user's views and needs for the product by providing a good user interface and interactive experience, and then use machine learning algorithms to improve and optimize the product. This iterative optimization process can continuously improve the quality and performance of the product to meet the changing needs of users. At the same time, designers need to consider how these algorithms and tools can be combined with interactive systems to provide a good user experience. This paper explores the potential applications of large language models, machine learning and interaction design for user interaction in recommendation systems and operating systems. By integrating these technologies, more intelligent and personalized services can be provided to meet user needs and promote continuous improvement and optimization of products. This is of great value for both recommendation research and user experience applications.

Data Pipeline plays an indispensable role in tasks such as modeling machine learning and developing data products. With the increasing diversification and complexity of Data sources, as well as the rapid growth of data volumes, building an efficient Data Pipeline has become crucial for improving work efficiency and solving complex problems. This paper focuses on exploring how to optimize data flow through automated machine learning methods by integrating AutoML with Data Pipeline. We will discuss how to leverage AutoML technology to enhance the intelligence of Data Pipeline, thereby achieving better results in machine learning tasks. By delving into the automation and optimization of Data flows, we uncover key strategies for constructing efficient data pipelines that can adapt to the ever-changing data landscape. This not only accelerates the modeling process but also provides innovative solutions to complex problems, enabling more significant outcomes in increasingly intricate data domains. Keywords- Data Pipeline Training;AutoML; Data environment; Machine learning

Pairwise matching cost aggregation is a crucial step for modern learning-based Multi-view Stereo (MVS). Prior works adopt an early aggregation scheme, which adds up pairwise costs into an intermediate cost. However, we analyze that this process can degrade informative pairwise matchings, thereby blocking the depth network from fully utilizing the original geometric matching cues. To address this challenge, we present a late aggregation approach that allows for aggregating pairwise costs throughout the network feed-forward process, achieving accurate estimations with only minor changes of the plain CasMVSNet. Instead of building an intermediate cost by weighted sum, late aggregation preserves all pairwise costs along a distinct view channel. This enables the succeeding depth network to fully utilize the crucial geometric cues without loss of cost fidelity. Grounded in the new aggregation scheme, we propose further techniques addressing view order dependence inside the preserved cost, handling flexible testing views, and improving the depth filtering process. Despite its technical simplicity, our method improves significantly upon the baseline cascade-based approach, achieving comparable results with state-of-the-art methods with favorable computation overhead., Comment: Code and models are available at https://github.com/Wuuu3511/LAMVSNET

Advanced change detection techniques primarily target image pairs of equal and high quality. However, variations in imaging conditions and platforms frequently lead to image pairs with distinct qualities: one image being high-quality, while the other being low-quality. These disparities in image quality present significant challenges for understanding image pairs semantically and extracting change features, ultimately resulting in a notable decline in performance. To tackle this challenge, we introduce an innovative training strategy grounded in knowledge distillation. The core idea revolves around leveraging task knowledge acquired from high-quality image pairs to guide the model's learning process when dealing with image pairs that exhibit differences in quality. Additionally, we develop a hierarchical correlation distillation approach (involving self-correlation, cross-correlation, and global correlation). This approach compels the student model to replicate the correlations inherent in the teacher model, rather than focusing solely on individual features. This ensures effective knowledge transfer while maintaining the student model's training flexibility., Comment: accepted by TGRS

The generalized $k$-connectivity of a graph $G$, denoted by $\kappa_k(G)$, is the minimum number of internally edge disjoint $S$-trees for any $S\subseteq V(G)$ and $|S|=k$. The generalized $k$-connectivity is a natural extension of the classical connectivity and plays a key role in applications related to the modern interconnection networks. An $n$-dimensional burnt pancake graph $BP_n$ is a Cayley graph which posses many desirable properties. In this paper, we try to evaluate the reliability of $BP_n$ by investigating its generalized 4-connectivity. By introducing the notation of inclusive tree and by studying structural properties of $BP_n$, we show that $\kappa_4(BP_n)=n-1$ for $n\ge 2$, that is, for any four vertices in $BP_n$, there exist ($n-1$) internally edge disjoint trees connecting them in $BP_n$.

In this paper, we consider the stochastic averaging principle and stability for multi-valued McKean-Vlasov stochastic differential equations with jumps. First, under certain averaging conditions, we are able to show that the solutions of the equations concerned can be approximated by solutions of the associated averaged multi-valued McKean-Vlasov stochastic differential equations with jumps in the sense of the mean square convergence. Second, we extend the classical It\^{o}'s formula from stochastic differential equations to multi-valued McKean-Vlasov stochastic differential equations with jumps. Last, as application of It\^{o}'s formula, we present the exponential stability of second moments, the exponentially 2-ultimate boundedness and the almost surely asymptotic stability for their solutions in terms of a Lyapunov function., Comment: 31 pages. arXiv admin note: text overlap with arXiv:2106.12080 by other authors

Modeling each hit as a multivariate event in racket sports and conducting sequential analysis aids in assessing player/team performance and identifying successful tactics for coaches and analysts. However, the complex correlations among multiple event attributes require pattern mining algorithms to be highly effective and efficient. This paper proposes Beep to discover meaningful multivariate patterns in racket sports. In particular, Beep introduces a new encoding scheme to discover patterns with correlations among multiple attributes and high-level tolerances of noise. Moreover, Beep applies an algorithm based on LSH (Locality-Sensitive Hashing) to accelerate summarizing patterns. We conducted a case study on a table tennis dataset and quantitative experiments on multi-scaled synthetic datasets to compare Beep with the SOTA multivariate pattern mining algorithm. Results showed that Beep can effectively discover patterns and noises to help analysts gain insights. Moreover, Beep was about five times faster than the SOTA algorithm., Comment: This work is accepted in the KDD 2023 Workshop on Data Science and AI for Sports

Different from conventional federated learning, personalized federated learning (PFL) is able to train a customized model for each individual client according to its unique requirement. The mainstream approach is to adopt a kind of weighted aggregation method to generate personalized models, in which weights are determined by the loss value or model parameters among different clients. However, such kinds of methods require clients to download others' models. It not only sheer increases communication traffic but also potentially infringes data privacy. In this paper, we propose a new PFL algorithm called \emph{FedDWA (Federated Learning with Dynamic Weight Adjustment)} to address the above problem, which leverages the parameter server (PS) to compute personalized aggregation weights based on collected models from clients. In this way, FedDWA can capture similarities between clients with much less communication overhead. More specifically, we formulate the PFL problem as an optimization problem by minimizing the distance between personalized models and guidance models, so as to customize aggregation weights for each client. Guidance models are obtained by the local one-step ahead adaptation on individual clients. Finally, we conduct extensive experiments using five real datasets and the results demonstrate that FedDWA can significantly reduce the communication traffic and achieve much higher model accuracy than the state-of-the-art approaches., Comment: Accepted by IJCAI 2023, camera ready version with appendix