Search

Collaborative perception can substantially boost each agent's perception ability by facilitating communication among multiple agents. However, temporal asynchrony among agents is inevitable in the real world due to communication delays, interruptions, and clock misalignments. This issue causes information mismatch during multi-agent fusion, seriously shaking the foundation of collaboration. To address this issue, we propose CoBEVFlow, an asynchrony-robust collaborative perception system based on bird's eye view (BEV) flow. The key intuition of CoBEVFlow is to compensate motions to align asynchronous collaboration messages sent by multiple agents. To model the motion in a scene, we propose BEV flow, which is a collection of the motion vector corresponding to each spatial location. Based on BEV flow, asynchronous perceptual features can be reassigned to appropriate positions, mitigating the impact of asynchrony. CoBEVFlow has two advantages: (i) CoBEVFlow can handle asynchronous collaboration messages sent at irregular, continuous time stamps without discretization; and (ii) with BEV flow, CoBEVFlow only transports the original perceptual features, instead of generating new perceptual features, avoiding additional noises. To validate CoBEVFlow's efficacy, we create IRregular V2V(IRV2V), the first synthetic collaborative perception dataset with various temporal asynchronies that simulate different real-world scenarios. Extensive experiments conducted on both IRV2V and the real-world dataset DAIR-V2X show that CoBEVFlow consistently outperforms other baselines and is robust in extremely asynchronous settings. The code is available at https://github.com/MediaBrain-SJTU/CoBEVFlow., Comment: 16 pages, 9 figures. Accepted by NeurIPS 2023

With the acceleration of urbanization, traffic forecasting has become an essential role in smart city construction. In the context of spatio-temporal prediction, the key lies in how to model the dependencies of sensors. However, existing works basically only consider the micro relationships between sensors, where the sensors are treated equally, and their macroscopic dependencies are neglected. In this paper, we argue to rethink the sensor's dependency modeling from two hierarchies: regional and global perspectives. Particularly, we merge original sensors with high intra-region correlation as a region node to preserve the inter-region dependency. Then, we generate representative and common spatio-temporal patterns as global nodes to reflect a global dependency between sensors and provide auxiliary information for spatio-temporal dependency learning. In pursuit of the generality and reality of node representations, we incorporate a Meta GCN to calibrate the regional and global nodes in the physical data space. Furthermore, we devise the cross-hierarchy graph convolution to propagate information from different hierarchies. In a nutshell, we propose a Hierarchical Information Enhanced Spatio-Temporal prediction method, HIEST, to create and utilize the regional dependency and common spatio-temporal patterns. Extensive experiments have verified the leading performance of our HIEST against state-of-the-art baselines. We publicize the code to ease reproducibility., Comment: 9 pages, accepted by CIKM'23

In the era of information explosion, spatio-temporal data mining serves as a critical part of urban management. Considering the various fields demanding attention, e.g., traffic state, human activity, and social event, predicting multiple spatio-temporal attributes simultaneously can alleviate regulatory pressure and foster smart city construction. However, current research can not handle the spatio-temporal multi-attribute prediction well due to the complex relationships between diverse attributes. The key challenge lies in how to address the common spatio-temporal patterns while tackling their distinctions. In this paper, we propose an effective solution for spatio-temporal multi-attribute prediction, PromptST. We devise a spatio-temporal transformer and a parameter-sharing training scheme to address the common knowledge among different spatio-temporal attributes. Then, we elaborate a spatio-temporal prompt tuning strategy to fit the specific attributes in a lightweight manner. Through the pretrain and prompt tuning phases, our PromptST is able to enhance the specific spatio-temoral characteristic capture by prompting the backbone model to fit the specific target attribute while maintaining the learned common knowledge. Extensive experiments on real-world datasets verify that our PromptST attains state-of-the-art performance. Furthermore, we also prove PromptST owns good transferability on unseen spatio-temporal attributes, which brings promising application potential in urban computing. The implementation code is available to ease reproducibility.

Consider the solutions $u$ to the elliptic equation $\mathcal{L}(u) = \partial_i(a^{ij}(x) \partial_j u) + b^i(x) \partial_i u + c(x) u= 0$ with $a^{ij}$ assumed only to be H\"older continuous. In this paper we prove an explicit bound for $(n-2)$-dimensional Minkowski estimates of singular set $\mathcal{S}(u) = \{ x \in B_1 : u(x) = |\nabla u(x)| = 0\}$ and critical set $\mathcal{C}(u) \equiv \{ x\in B_{1} : |\nabla u(x)| = 0 \}$ in terms of the bound on doubling index, depending on $c \equiv 0$ or not. Here the H\"older assumption is sharp as it is the weakest condition in order to define the critical set of $u$ according to elliptic estimates. We can also obtain an optimal improvement on Cheeger-Naber-Valtorta's volume estimates on each quantitative stratum $\mathcal{S}^k_{\eta, r}$. The main difficulty in this situation is the lack of monotonicity formula which is essential to the quantitative stratification. In our proof, one key ingredient is a new almost monotonicity formula for doubling index under the H\"older assumption. Another key ingredient is the quantitative uniqueness of tangent maps. It deserves to note that our almost monotonicity is sufficient to address all the difficulties arising from the absence of monotonicity in the analysis of differential equations. We believe the idea could be applied to other relevant study.

We show that one-dimensional Floquet trimer arrays with periodically oscillating waveguides support two different and co-existing types of topological Floquet edge states in two different topological gaps in Floquet spectrum. In these systems nontrivial topology is introduced by longitudinal periodic oscillations of the waveguide centers, leading to the formation of Floquet edge states in certain range of oscillation amplitudes despite the fact that the structure spends half of the period in ``instantaneously'' nontopological phase, and only during other half-period it is ``instantaneously'' topological. Two co-existing Floquet edge states are characterized by different phase relations between bright spots in the unit cell -- in one mode these spots are in-phase, while in other mode they are out-of-phase. We show that in focusing nonlinear medium topological Floquet edge solitons, representing exactly periodic nonlinear localized Floquet states, can bifurcate from both these types of linear edge states. Both types of Floquet edge solitons can be stable and can be created dynamically using two-site excitations., Comment: 9 pages, 6 figures

VQA Natural Language Explanation (VQA-NLE) task aims to explain the decision-making process of VQA models in natural language. Unlike traditional attention or gradient analysis, free-text rationales can be easier to understand and gain users' trust. Existing methods mostly use post-hoc or self-rationalization models to obtain a plausible explanation. However, these frameworks are bottlenecked by the following challenges: 1) the reasoning process cannot be faithfully responded to and suffer from the problem of logical inconsistency. 2) Human-annotated explanations are expensive and time-consuming to collect. In this paper, we propose a new Semi-Supervised VQA-NLE via Self-Critical Learning (S3C), which evaluates the candidate explanations by answering rewards to improve the logical consistency between answers and rationales. With a semi-supervised learning framework, the S3C can benefit from a tremendous amount of samples without human-annotated explanations. A large number of automatic measures and human evaluations all show the effectiveness of our method. Meanwhile, the framework achieves a new state-of-the-art performance on the two VQA-NLE datasets., Comment: CVPR2023

A central challenge of video prediction lies where the system has to reason the objects' future motions from image frames while simultaneously maintaining the consistency of their appearances across frames. This work introduces an end-to-end trainable two-stream video prediction framework, Motion-Matrix-based Video Prediction (MMVP), to tackle this challenge. Unlike previous methods that usually handle motion prediction and appearance maintenance within the same set of modules, MMVP decouples motion and appearance information by constructing appearance-agnostic motion matrices. The motion matrices represent the temporal similarity of each and every pair of feature patches in the input frames, and are the sole input of the motion prediction module in MMVP. This design improves video prediction in both accuracy and efficiency, and reduces the model size. Results of extensive experiments demonstrate that MMVP outperforms state-of-the-art systems on public data sets by non-negligible large margins (about 1 db in PSNR, UCF Sports) in significantly smaller model sizes (84% the size or smaller)., Comment: ICCV 2023 (Oral)

Boolean Satisfiability (SAT) and Satisfiability Modulo Theories (SMT) are widely used in automated verification, but there is a lack of interactive tools designed for educational purposes in this field. To address this gap, we present EduSAT, a pedagogical tool specifically developed to support learning and understanding of SAT and SMT solving. EduSAT offers implementations of key algorithms such as the Davis-Putnam-Logemann-Loveland (DPLL) algorithm and the Reduced Order Binary Decision Diagram (ROBDD) for SAT solving. Additionally, EduSAT provides solver abstractions for five NP-complete problems beyond SAT and SMT. Users can benefit from EduSAT by experimenting, analyzing, and validating their understanding of SAT and SMT solving techniques. Our tool is accompanied by comprehensive documentation and tutorials, extensive testing, and practical features such as a natural language interface and SAT and SMT formula generators, which also serve as a valuable opportunity for learners to deepen their understanding. Our evaluation of EduSAT demonstrates its high accuracy, achieving 100% correctness across all the implemented SAT and SMT solvers. We release EduSAT as a python package in .whl file, and the source can be identified at https://github.com/zhaoy37/SAT_Solver.

The electron bubble in superfluid helium has two degrees of freedom that may offer exceptionally low dissipation: the electron's spin and the bubble's motion. If these degrees of freedom can be read out and controlled with sufficient sensitivity, they would provide a novel platform for realizing a range of quantum technologies and for exploring open questions in the physics of superfluid helium. Here we propose a practical scheme for accomplishing this by trapping an electron bubble inside a superfluid-filled opto-acoustic cavity., Comment: Main text: 5 pages, 5 figures. Supplement: 13 pages, 4 figures, 1 table

High harmonic generation (HHG) with longitudinal optical orbital angular momentum has attracted much attention over the past decade. Here, we present the first study on the HHG with transverse orbital angular momentum driven by the spatiotemporal optical vortex (STOV) pulses. We show that the produced spatial resolved harmonic spectra reveal unique structures, such as the spatially spectral tilt and the fine interference patterns. We show these spatio-spectral structures originate from both the macroscopic and microscopic effect of spatiotemporal optical singularity in HHG. Employing two-color counter-spin and counter-vorticity STOV pulses, we further discuss a robust method to control the spatiotemporal topological charge and spectral structure of high-order harmonics. The conservation rule of photon transverse orbital angular momentum in HHG process is also discussed when mixing with photon spin angular momenta., Comment: 15 pages, 5 figures

Optical skyrmion serves as a crucial interface between optics and topology. Recently, it has attracted great interest in linear optics. Here, we theoretically introduce a framework for the all-optical generation and control of freespace optical skyrmions in extreme ultraviolet regions via high harmonic generation. We show that by employing full Poincare beams, the created extreme ultraviolet fields manifest as skyrmionic structures in Stokes vector fields, whose skyrmion number is relevant to harmonic orders. We reveal that the generation of skyrmionics structure is attributed to spatial-resolved spin constraint of high harmonic generation. Through qualifying the geometrical parameters of full Poincar\'e beams, the topological texture of extreme ultraviolet fields can be completely manipulated, generating the Bloch-type, Neel-type, anti-type, and higher-order skyrmions, etc. This work promotes the investigation of topological optics in optical highly nonlinear processes, with potential applications towards ultrafast spintronics with structured light fields., Comment: 17 Pages, 4 figures

The interaction between a quantum particle's spin angular momentum and its orbital angular momentum is ubiquitous in nature. In optics, the spin-orbit optical phenomenon is closely related with the light-matter interaction and has been of great interest. With the development of laser technology, the high-power and ultrafast light sources now serve as a crucial tool in revealing the behaviour of matters under extreme conditions. The comprehensive knowledge of the spin-orbit interaction for the intense light is of utmost importance. Here, we achieve the in-situ modulation and visualization of the optical orbital-to-spin conversion in strong-field regime. We show that, through manipulating the morphology of femtosecond cylindrical vector vortex pulses by a slit, the photons' orbital angular momentum can be controllably transformed into spin after focusing. By employing strong-field ionization experiment, the orbital-to-spin conversion can be imaged and measured through the photoelectron momentum distributions. Such detection and consequent control of spin-orbit dynamics of intense laser fields have implications on controlling the photoelectron holography and coherent extreme ultraviolet radiation., Comment: 28 pages, 4 figures

The explicit neural radiance field (NeRF) has gained considerable interest for its efficient training and fast inference capabilities, making it a promising direction such as virtual reality and gaming. In particular, PlenOctree (POT)[1], an explicit hierarchical multi-scale octree representation, has emerged as a structural and influential framework. However, POT's fixed structure for direct optimization is sub-optimal as the scene complexity evolves continuously with updates to cached color and density, necessitating refining the sampling distribution to capture signal complexity accordingly. To address this issue, we propose the dynamic PlenOctree DOT, which adaptively refines the sample distribution to adjust to changing scene complexity. Specifically, DOT proposes a concise yet novel hierarchical feature fusion strategy during the iterative rendering process. Firstly, it identifies the regions of interest through training signals to ensure adaptive and efficient refinement. Next, rather than directly filtering out valueless nodes, DOT introduces the sampling and pruning operations for octrees to aggregate features, enabling rapid parameter learning. Compared with POT, our DOT outperforms it by enhancing visual quality, reducing over $55.15$/$68.84\%$ parameters, and providing 1.7/1.9 times FPS for NeRF-synthetic and Tanks $\&$ Temples, respectively. Project homepage:https://vlislab22.github.io/DOT. [1] Yu, Alex, et al. "Plenoctrees for real-time rendering of neural radiance fields." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021., Comment: Accepted by ICCV2023

Correlated sampling has wide-ranging applications in Monte Carlo calculations. When branching random walks are involved, as commonly found in many algorithms in quantum physics and electronic structure, population control is typically not applied with correlated sampling due to technical difficulties. This hinders the stability and efficiency of correlated sampling. In this work, we study schemes for allowing birth/death in correlated sampling and propose an algorithm for population control. The algorithm can be realized in several variants depending on the application. One variant is a static method that creates a reference run and allows other correlated calculations to be added a posteriori. Another optimizes the population control for a set of correlated, concurrent runs dynamically. These approaches are tested in different applications in quantum systems, including both the Hubbard model and electronic structure calculations in real materials., Comment: 10 pages, 8 figures

Floquet systems with periodically varying in time parameters enable realization of unconventional topological phases that do not exist in static systems with constant parameters and that are frequently accompanied by appearance of novel types of the topological states. Among such Floquet systems are the Su-Schrieffer-Heeger lattices with periodically-modulated couplings that can support at their edges anomalous $\pi$ modes of topological origin despite the fact that the lattice spends only half of the evolution period in topologically nontrivial phase, while during other half-period it is topologically trivial. Here, using Su-Schrieffer-Heeger arrays composed from periodically oscillating waveguides inscribed in transparent nonlinear optical medium, we report experimental observation of photonic anomalous $\pi$ modes residing at the edge or in the corner of the one- or two-dimensional arrays, respectively, and demonstrate a new class of topological $\pi$ solitons bifurcating from such modes in the topological gap of the Floquet spectrum at high powers. $\pi$ solitons reported here are strongly oscillating nonlinear Floquet states exactly reproducing their profiles after each longitudinal period of the structure. They can be dynamically stable in both one- and two-dimensional oscillating waveguide arrays, the latter ones representing the first realization of the Floquet photonic higher-order topological insulator, while localization properties of such $\pi$ solitons are determined by their power., Comment: 10 pages, 6 figures, to appear in Science Bulletin

Data driven transmission line fault location methods have the potential to more accurately locate faults by extracting fault information from available data. However, most of the data driven fault location methods in the literature are not validated by field data for the following reasons. On one hand, the available field data during faults are very limited for one specific transmission line, and using field data for training is close to impossible. On the other hand, if simulation data are utilized for training, the mismatch between the simulation system and the practical system will cause fault location errors. To this end, this paper proposes a physics-informed data-driven fault location method. The data from a practical fault event are first analyzed to extract the ranges of system and fault parameters such as equivalent source impedances, loading conditions, fault inception angles (FIA) and fault resistances. Afterwards, the simulation system is constructed with the ranges of parameters, to generate data for training. This procedure merges the gap between simulation and practical power systems, and at the same time considers the uncertainty of system and fault parameters in practice. The proposed data-driven method does not require system parameters, only requires instantaneous voltage and current measurements at the local terminal, with a low sampling rate of several kHz and a short fault time window of half a cycle before and after the fault occurs. Numerical experiments and field data experiments clearly validate the advantages of the proposed method over existing data driven methods., Comment: 10 pages, 27 figures

High-contrast scattering problems are special among classical wave systems as they allow for strong wave focusing and localisation at low frequencies. We use an asymptotic framework to develop a landscape theory for high-contrast systems that resonate in a subwavelength regime. Our from-first-principles asymptotic analysis yields a characterisation in terms of the generalised capacitance matrix, giving a discrete approximation of the three-dimensional scattering problem. We develop landscape theory for the generalised capacitance matrix and use it to predict the positions of three-dimensional wave focusing and localisation in random and non-periodic systems of subwavelength resonators.

With the prosperity of e-commerce and web applications, Recommender Systems (RecSys) have become an important component of our daily life, providing personalized suggestions that cater to user preferences. While Deep Neural Networks (DNNs) have made significant advancements in enhancing recommender systems by modeling user-item interactions and incorporating textual side information, DNN-based methods still face limitations, such as difficulties in understanding users' interests and capturing textual side information, inabilities in generalizing to various recommendation scenarios and reasoning on their predictions, etc. Meanwhile, the emergence of Large Language Models (LLMs), such as ChatGPT and GPT4, has revolutionized the fields of Natural Language Processing (NLP) and Artificial Intelligence (AI), due to their remarkable abilities in fundamental responsibilities of language understanding and generation, as well as impressive generalization and reasoning capabilities. As a result, recent studies have attempted to harness the power of LLMs to enhance recommender systems. Given the rapid evolution of this research direction in recommender systems, there is a pressing need for a systematic overview that summarizes existing LLM-empowered recommender systems, to provide researchers in relevant fields with an in-depth understanding. Therefore, in this paper, we conduct a comprehensive review of LLM-empowered recommender systems from various aspects including Pre-training, Fine-tuning, and Prompting. More specifically, we first introduce representative methods to harness the power of LLMs (as a feature encoder) for learning representations of users and items. Then, we review recent techniques of LLMs for enhancing recommender systems from three paradigms, namely pre-training, fine-tuning, and prompting. Finally, we comprehensively discuss future directions in this emerging field., Comment: Accepted by IEEE TKDE