Search

Pre-training & fine-tuning can enhance the transferring efficiency and performance in visual tasks. Recent delta-tuning methods provide more options for visual classification tasks. Despite their success, existing visual delta-tuning art fails to exceed the upper limit of full fine-tuning on challenging tasks like object detection and segmentation. To find a competitive alternative to full fine-tuning, we propose the Multi-cognitive Visual Adapter (Mona) tuning, a novel adapter-based tuning method. First, we introduce multiple vision-friendly filters into the adapter to enhance its ability to process visual signals, while previous methods mainly rely on language-friendly linear filters. Second, we add the scaled normalization layer in the adapter to regulate the distribution of input features for visual filters. To fully demonstrate the practicality and generality of Mona, we conduct experiments on multiple representative visual tasks, including instance segmentation on COCO, semantic segmentation on ADE20K, object detection on Pascal VOC, oriented object detection on DOTA/STAR, and image classification on three common datasets. Exciting results illustrate that Mona surpasses full fine-tuning on all these tasks, and is the only delta-tuning method outperforming full fine-tuning on the above various tasks. For example, Mona achieves 1% performance gain on the COCO dataset compared to full fine-tuning. Comprehensive results suggest that Mona-tuning is more suitable for retaining and utilizing the capabilities of pre-trained models than full fine-tuning. The code will be released at https://github.com/Leiyi-Hu/mona., Comment: arXiv admin note: substantial text overlap with arXiv:2311.15010

In this paper, we establish the averaging principle for stochastic functional partial differential equations (SFPDEs) characterized by H\"older coefficients and infinite delay. Firstly, we rigorously establish the existence and uniqueness of strong solutions for a specific class of finite-dimensional systems characterized by H\"older continuous coefficients and infinite delay. We extend these results to their infinite-dimensional counterparts using the variational approach and Galerkin projection technique. Subsequently, we establish the averaging principle (the first Bogolyubov theorem) for SFPDEs with infinite delay, subject to conditions of linear growth and H\"older continuity. This is achieved through classical Khasminskii time discretization and reductio ad absurdum, illustrating the convergence of solutions from the original Cauchy problem to those of the averaged equation across the finite interval [0, T]. To illustrate our findings, we present two applications: stochastic generalized porous media equations and stochastic reaction-diffusion equations with H\"older coefficients.

In this paper, we study the problem of Poisson stability of solutions for stochastic semi-linear evolution equation driven by fractional Brownian motion \mathrm{d} X(t)= \left( AX(t) + f(t, X(t)) \right) \mathrm{d}t + g\left(t, X(t)\right)\mathrm{d}B^H_{Q}(t), where A is an exponentially stable linear operator acting on a separable Hilbert space \mathbb{H}, coefficients f and g are Poisson stable in time, and B^H_Q (t) is a Q-cylindrical fBm with Hurst index H. First, we establish the existence and uniqueness of the solution for this equation. Then, we prove that under the condition where the functions f and g are sufficiently "small", the equation admits a solution that exhibits the same character of recurrence as f and g. The discussion is further extended to the asymptotic stability of these Poisson stable solutions. Finally, we include an example to validate our results., Comment: 25 pages, 1 figure. arXiv admin note: substantial text overlap with arXiv:1702.02718 by other authors

Thermodynamics as a fundamental branch of physics examines the relationships between heat, work, and energy. The maximum energy extraction can be characterized by using the passive states that has no extracted energy under any cyclic unitary process. In this paper, we focus on the concept of marginal passive state energy and derive polygon inequalities for multi-qubit entangled pure states. We show that the marginal passive state energies collectively form a convex polytope for each class of quantum states that are equivalent under SLOCC. We finally introduce multipartite passive state energy criteria to classify multipartite entanglement under SLOCC. The present result provides a thermodynamic method to witness multipartite entanglement., Comment: 7 pages. We have corrected the abstract. arXiv admin note: text overlap with arXiv:2406.13319

The presence of quantum multipartite entanglement implies the existence of a thermodynamic quantity known as the ergotropic gap, which is defined as the difference between the maximal global and local extractable works from the system. We establish a direct relation between the geometric measure of entanglement and the ergotropic gaps. We show that all the marginal ergotropic gaps form a convex polytope for each class of quantum states that are equivalent under stochastic local operations and classical communication (SLOCC). We finally introduce the concept of multipartite ergotropic gap indicators and use them to present a refined criterion for classifying entanglement under SLOCC., Comment: 7 pages (to be published)

Scene graph generation (SGG) in satellite imagery (SAI) benefits promoting understanding of geospatial scenarios from perception to cognition. In SAI, objects exhibit great variations in scales and aspect ratios, and there exist rich relationships between objects (even between spatially disjoint objects), which makes it attractive to holistically conduct SGG in large-size very-high-resolution (VHR) SAI. However, there lack such SGG datasets. Due to the complexity of large-size SAI, mining triplets heavily relies on long-range contextual reasoning. Consequently, SGG models designed for small-size natural imagery are not directly applicable to large-size SAI. This paper constructs a large-scale dataset for SGG in large-size VHR SAI with image sizes ranging from 512 x 768 to 27,860 x 31,096 pixels, named STAR (Scene graph generaTion in lArge-size satellite imageRy), encompassing over 210K objects and over 400K triplets. To realize SGG in large-size SAI, we propose a context-aware cascade cognition (CAC) framework to understand SAI regarding object detection (OBD), pair pruning and relationship prediction for SGG. We also release a SAI-oriented SGG toolkit with about 30 OBD and 10 SGG methods which need further adaptation by our devised modules on our challenging STAR dataset. The dataset and toolkit are available at: https://linlin-dev.github.io/project/STAR., Comment: 18 pages, 11 figures

Vision-Language Foundation Models (VLFMs) have made remarkable progress on various multimodal tasks, such as image captioning, image-text retrieval, visual question answering, and visual grounding. However, most methods rely on training with general image datasets, and the lack of geospatial data leads to poor performance on earth observation. Numerous geospatial image-text pair datasets and VLFMs fine-tuned on them have been proposed recently. These new approaches aim to leverage large-scale, multimodal geospatial data to build versatile intelligent models with diverse geo-perceptive capabilities, which we refer to as Vision-Language Geo-Foundation Models (VLGFMs). This paper thoroughly reviews VLGFMs, summarizing and analyzing recent developments in the field. In particular, we introduce the background and motivation behind the rise of VLGFMs, highlighting their unique research significance. Then, we systematically summarize the core technologies employed in VLGFMs, including data construction, model architectures, and applications of various multimodal geospatial tasks. Finally, we conclude with insights, issues, and discussions regarding future research directions. To the best of our knowledge, this is the first comprehensive literature review of VLGFMs. We keep tracing related works at https://github.com/zytx121/Awesome-VLGFM., Comment: 18 pages, 4 figures

Multi-UAV collaborative 3D object detection can perceive and comprehend complex environments by integrating complementary information, with applications encompassing traffic monitoring, delivery services and agricultural management. However, the extremely broad observations in aerial remote sensing and significant perspective differences across multiple UAVs make it challenging to achieve precise and consistent feature mapping from 2D images to 3D space in multi-UAV collaborative 3D object detection paradigm. To address the problem, we propose an unparalleled camera-based multi-UAV collaborative 3D object detection paradigm called UCDNet. Specifically, the depth information from the UAVs to the ground is explicitly utilized as a strong prior to provide a reference for more accurate and generalizable feature mapping. Additionally, we design a homologous points geometric consistency loss as an auxiliary self-supervision, which directly influences the feature mapping module, thereby strengthening the global consistency of multi-view perception. Experiments on AeroCollab3D and CoPerception-UAVs datasets show our method increases 4.7% and 10% mAP respectively compared to the baseline, which demonstrates the superiority of UCDNet.

Image pyramids are commonly used in modern computer vision tasks to obtain multi-scale features for precise understanding of images. However, image pyramids process multiple resolutions of images using the same large-scale model, which requires significant computational cost. To overcome this issue, we propose a novel network architecture known as the Parameter-Inverted Image Pyramid Networks (PIIP). Our core idea is to use models with different parameter sizes to process different resolution levels of the image pyramid, thereby balancing computational efficiency and performance. Specifically, the input to PIIP is a set of multi-scale images, where higher resolution images are processed by smaller networks. We further propose a feature interaction mechanism to allow features of different resolutions to complement each other and effectively integrate information from different spatial scales. Extensive experiments demonstrate that the PIIP achieves superior performance in tasks such as object detection, segmentation, and image classification, compared to traditional image pyramid methods and single-branch networks, while reducing computational cost. Notably, when applying our method on a large-scale vision foundation model InternViT-6B, we improve its performance by 1%-2% on detection and segmentation with only 40%-60% of the original computation. These results validate the effectiveness of the PIIP approach and provide a new technical direction for future vision computing tasks. Our code and models are available at https://github.com/OpenGVLab/PIIP.

Adapting pre-trained foundation models for various downstream tasks has been prevalent in artificial intelligence. Due to the vast number of tasks and high costs, adjusting all parameters becomes unfeasible. To mitigate this, several fine-tuning techniques have been developed to update the pre-trained model weights in a more resource-efficient manner, such as through low-rank adjustments. Yet, almost all of these methods focus on linear weights, neglecting the intricacies of parameter spaces in higher dimensions like 4D. Alternatively, some methods can be adapted for high-dimensional parameter space by compressing changes in the original space into two dimensions and then employing low-rank matrix decomposition. However, these approaches destructs the structural integrity of the involved high-dimensional spaces. To tackle the diversity of dimensional spaces across different foundation models and provide a more precise representation of the changes within these spaces, this paper introduces a generalized parameter-efficient fine-tuning framework, FLoRA, designed for various dimensional parameter space. Specifically, utilizing Tucker decomposition, FLoRA asserts that changes in each dimensional parameter space are based on a low-rank core space which maintains the consistent topological structure with the original space. It then models the changes through this core space alongside corresponding weights to reconstruct alterations in the original space. FLoRA effectively preserves the structural integrity of the change of original N-dimensional parameter space, meanwhile decomposes it via low-rank tensor decomposition. Extensive experiments on computer vision, natural language processing and multi-modal tasks validate FLoRA's effectiveness. Codes are available at https://github.com/SJTU-DeepVisionLab/FLoRA.

Collaborative trajectory prediction can comprehensively forecast the future motion of objects through multi-view complementary information. However, it encounters two main challenges in multi-drone collaboration settings. The expansive aerial observations make it difficult to generate precise Bird's Eye View (BEV) representations. Besides, excessive interactions can not meet real-time prediction requirements within the constrained drone-based communication bandwidth. To address these problems, we propose a novel framework named "Drones Help Drones" (DHD). Firstly, we incorporate the ground priors provided by the drone's inclined observation to estimate the distance between objects and drones, leading to more precise BEV generation. Secondly, we design a selective mechanism based on the local feature discrepancy to prioritize the critical information contributing to prediction tasks during inter-drone interactions. Additionally, we create the first dataset for multi-drone collaborative prediction, named "Air-Co-Pred", and conduct quantitative and qualitative experiments to validate the effectiveness of our DHD framework.The results demonstrate that compared to state-of-the-art approaches, DHD reduces position deviation in BEV representations by over 20% and requires only a quarter of the transmission ratio for interactions while achieving comparable prediction performance. Moreover, DHD also shows promising generalization to the collaborative 3D object detection in CoPerception-UAVs.

This paper examines Poisson stable (including stationary, periodic, almost periodic, Levitan almost periodic, Bohr almost automorphic, pseudo-periodic, Birkhoff recurrent, pseudo-recurrent, etc.) measures and limit theorems for stochastic functional partial differential equations(SFPDEs) with monotone coefficients. We first show the existence and uniqueness of entrance measure $\mu _{t}$ for SFPDEs by dissipative method (or remoting start). Then, with the help of Shcherbakov's comparability method in character of recurrence, we prove that the entrance measure inherits the same recurrence of coefficients. Thirdly, we show the tightness of the set of measures $\mu _{t}$. As a result, any sequence of the average of $\{\mu _{t}\}_{t\in\mathbb{R} }$ have the limit point $\mu ^{*}$. Further, we study the uniform exponential mixing of the measure $\mu ^{*}$ in the sense of Wasserstein metric. Fourthly, under uniform exponential mixing and Markov property, we establish the strong law of large numbers, the central limit theorem and estimate the corresponding rates of convergence for solution maps of SFPDEs. Finally, we give applications of stochastic generalized porous media equations with delay to illustrate of our results.

This paper investigates the existence and uniqueness of solutions, as well as the ergodicity and exponential mixing to invariant measures, and limit theorems for a class of McKean-Vlasov SPDEs characterized by H\"lder continuity. We rigorously establish the existence and uniqueness of strong solutions for a specific class of finite-dimensional systems with H\"older continuous coefficients. Extending these results to the infinite-dimensional counterparts using the Galerkin projection technique. Additionally, we explore the properties of the solutions, including time homogeneity, the Markov and the Feller property. Building upon these properties, we examine the exponential ergodicity and mixing of invariant measures under Lyapunov conditions. Finally, within the framework of coefficients meeting the criteria of H\"lder continuity and Lyapunov conditions, alongside the uniform mixing property of invariant measures, we establish the strong law of large numbers and the central limit theorem for the solution and obtain estimates of corresponding convergence rates.

Since synchrotron polarization fluctuations are related to the fundamental properties of the magnetic field, we propose the polarization intensity variance to measure the Galactic interstellar medium (ISM) magnetization. We confirm the method's applicability by comparing it with the polarization angle dispersion and its reliability by measuring the underlying Alfv\'enic Mach number of MHD turbulence. With the finding of the power-law relation of $\mathcal{A} \propto M_{\rm A}^{2}$ between polarization intensity variance $\mathcal{A}$ and Alfv\'enic Mach number $M_{\rm A}$, we apply the new technique to the Canadian Galactic Plane Survey (CGPS) data, achieving Alfv\'enic Mach number of the Galactic ISM. Our results show that the low-latitude Galactic ISM is dominated by sub-Alf\'enic turbulence, with $M_{\rm A}$ approximately between 0.5 and 1.0., Comment: 10 pages, 5 figures, and 2 tables. Accepted for publication in ApJ

In our urban life, Illegal Driver Substitution (IDS) activity for a taxi is a grave unlawful activity in the taxi industry, possibly causing severe traffic accidents and painful social repercussions. Currently, the IDS activity is manually supervised by law enforcers, i.e., law enforcers empirically choose a taxi and inspect it. The pressing problem of this scheme is the dilemma between the limited number of law-enforcers and the large volume of taxis. In this paper, motivated by this problem, we propose a computational method that helps law enforcers efficiently find the taxis which tend to have the IDS activity. Firstly, our method converts the identification of the IDS activity to a supervised learning task. Secondly, two kinds of taxi driver behaviors, i.e., the Sleeping Time and Location (STL) behavior and the Pick-Up (PU) behavior are proposed. Thirdly, the multiple scale pooling on self-similarity is proposed to encode the individual behaviors into the universal features for all taxis. Finally, a Multiple Component- Multiple Instance Learning (MC-MIL) method is proposed to handle the deficiency of the behavior features and to align the behavior features simultaneously. Extensive experiments on a real-world data set shows that the proposed behavior features have a good generalization ability across different classifiers, and the proposed MC-MIL method suppresses the baseline methods.

Quantum coherence plays a crucial role in manipulating and controlling quantum systems, leading to breakthroughs in various fields such as quantum information, quantum sensing, and the detection of gravitational waves. Most coherence witnesses rely on the assumption of being able to control quantum states. Here we report a device-independent coherence model by extending the standard Bell theory to multiple source scenarios. We propose a Greenberger-Horne-Zeilinger-type paradox to verify the particle and wave behaviors of a coherent carrier. We experimentally generate generalized two-photon entangled states that violate the present paradox, witnessing spatial quantum superposition through local measurements., Comment: 19 pages, 7 figures (Almost publish version)

We examine the properties of atomic current in a superfluid oscillating circuit consisting of a mesoscopic channel that connects two reservoirs of a Bose-Einstein condensate. We investigate the presence of a critical current in the channel and examine how the amplitude of the oscillations in the number imbalance between the two reservoirs varies with system parameters. In addition to highlighting that the dissipative resistance stems from the formation of vortex pairs, we also illustrate the role of these vortex pairs as a quantum current regulator. The dissipation strength is discrete based on the number imbalance, which corresponds to the emergence of vortex pairs in the system. Our findings indicate that the circuit demonstrates characteristics of both voltage-limiting and current-limiting mechanisms. To model the damping behavior of the atomic superfluid circuit, we develop an equivalent LC oscillator circuit with a quantum current regulator., Comment: 6 figures

Multimodal image fusion and object detection are crucial for autonomous driving. While current methods have advanced the fusion of texture details and semantic information, their complex training processes hinder broader applications. Addressing this challenge, we introduce E2E-MFD, a novel end-to-end algorithm for multimodal fusion detection. E2E-MFD streamlines the process, achieving high performance with a single training phase. It employs synchronous joint optimization across components to avoid suboptimal solutions tied to individual tasks. Furthermore, it implements a comprehensive optimization strategy in the gradient matrix for shared parameters, ensuring convergence to an optimal fusion detection configuration. Our extensive testing on multiple public datasets reveals E2E-MFD's superior capabilities, showcasing not only visually appealing image fusion but also impressive detection outcomes, such as a 3.9% and 2.0% mAP50 increase on horizontal object detection dataset M3FD and oriented object detection dataset DroneVehicle, respectively, compared to state-of-the-art approaches. The code is released at https://github.com/icey-zhang/E2E-MFD.

Considerable efforts have been devoted to Oriented Object Detection (OOD). However, one lasting issue regarding the discontinuity in Oriented Bounding Box (OBB) representation remains unresolved, which is an inherent bottleneck for extant OOD methods. This paper endeavors to completely solve this issue in a theoretically guaranteed manner and puts an end to the ad-hoc efforts in this direction. Prior studies typically can only address one of the two cases of discontinuity: rotation and aspect ratio, and often inadvertently introduce decoding discontinuity, e.g. Decoding Incompleteness (DI) and Decoding Ambiguity (DA) as discussed in literature. Specifically, we propose a novel representation method called Continuous OBB (COBB), which can be readily integrated into existing detectors e.g. Faster-RCNN as a plugin. It can theoretically ensure continuity in bounding box regression which to our best knowledge, has not been achieved in literature for rectangle-based object representation. For fairness and transparency of experiments, we have developed a modularized benchmark based on the open-source deep learning framework Jittor's detection toolbox JDet for OOD evaluation. On the popular DOTA dataset, by integrating Faster-RCNN as the same baseline model, our new method outperforms the peer method Gliding Vertex by 1.13% mAP50 (relative improvement 1.54%), and 2.46% mAP75 (relative improvement 5.91%), without any tricks., Comment: 17 pages, 12 tables, 8 figures. Accepted by CVPR'24. Code: https://github.com/514flowey/JDet-COBB

In target speaker extraction, many studies rely on the speaker embedding which is obtained from an enrollment of the target speaker and employed as the guidance. However, solely using speaker embedding may not fully utilize the contextual information contained in the enrollment. In this paper, we directly exploit this contextual information in the time-frequency (T-F) domain. Specifically, the T-F representations of the enrollment and the mixed signal are interacted to compute the weighting matrices through an attention mechanism. These weighting matrices reflect the similarity among different frames of the T-F representations and are further employed to obtain the consistent T-F representations of the enrollment. These consistent representations are served as the guidance, allowing for better exploitation of the contextual information. Furthermore, the proposed method achieves the state-of-the-art performance on the benchmark dataset and shows its effectiveness in the complex scenarios., Comment: Accepted by ICASSP 2024

Many reinforcement learning environments (e.g., Minecraft) provide only sparse rewards that indicate task completion or failure with binary values. The challenge in exploration efficiency in such environments makes it difficult for reinforcement-learning-based agents to learn complex tasks. To address this, this paper introduces an advanced learning system, named Auto MC-Reward, that leverages Large Language Models (LLMs) to automatically design dense reward functions, thereby enhancing the learning efficiency. Auto MC-Reward consists of three important components: Reward Designer, Reward Critic, and Trajectory Analyzer. Given the environment information and task descriptions, the Reward Designer first design the reward function by coding an executable Python function with predefined observation inputs. Then, our Reward Critic will be responsible for verifying the code, checking whether the code is self-consistent and free of syntax and semantic errors. Further, the Trajectory Analyzer summarizes possible failure causes and provides refinement suggestions according to collected trajectories. In the next round, Reward Designer will further refine and iterate the dense reward function based on feedback. Experiments demonstrate a significant improvement in the success rate and learning efficiency of our agents in complex tasks in Minecraft, such as obtaining diamond with the efficient ability to avoid lava, and efficiently explore trees and animals that are sparse in the plains biome., Comment: Accepted by CVPR2024

Existing scene text detection methods typically rely on extensive real data for training. Due to the lack of annotated real images, recent works have attempted to exploit large-scale labeled synthetic data (LSD) for pre-training text detectors. However, a synth-to-real domain gap emerges, further limiting the performance of text detectors. Differently, in this work, we propose FreeReal, a real-domain-aligned pre-training paradigm that enables the complementary strengths of both LSD and unlabeled real data (URD). Specifically, to bridge real and synthetic worlds for pre-training, a glyph-based mixing mechanism (GlyphMix) is tailored for text images.GlyphMix delineates the character structures of synthetic images and embeds them as graffiti-like units onto real images. Without introducing real domain drift, GlyphMix freely yields real-world images with annotations derived from synthetic labels. Furthermore, when given free fine-grained synthetic labels, GlyphMix can effectively bridge the linguistic domain gap stemming from English-dominated LSD to URD in various languages. Without bells and whistles, FreeReal achieves average gains of 1.97%, 3.90%, 3.85%, and 4.56% in improving the performance of FCENet, PSENet, PANet, and DBNet methods, respectively, consistently outperforming previous pre-training methods by a substantial margin across four public datasets. Code is available at https://github.com/SJTU-DeepVisionLab/FreeReal., Comment: Accepted by ECCV2024

The complexity and scale of IT systems are increasing dramatically, posing many challenges to real-world anomaly detection. Deep learning anomaly detection has emerged, aiming at feature learning and anomaly scoring, which has gained tremendous success. However, little work has been done on the thresholding problem despite it being a critical factor for the effectiveness of anomaly detection. In this paper, we model thresholding in anomaly detection as a Markov Decision Process and propose an agent-based dynamic thresholding (ADT) framework based on a deep Q-network. The proposed method can be integrated into many systems that require dynamic thresholding. An auto-encoder is utilized in this study to obtain feature representations and produce anomaly scores for complex input data. ADT can adjust thresholds adaptively by utilizing the anomaly scores from the auto-encoder and significantly improve anomaly detection performance. The properties of ADT are studied through experiments on three real-world datasets and compared with benchmarks, hence demonstrating its thresholding capability, data-efficient learning, stability, and robustness. Our study validates the effectiveness of reinforcement learning in optimal thresholding control in anomaly detection.

With the rapidly increasing demand for oriented object detection (OOD), recent research involving weakly-supervised detectors for learning rotated box (RBox) from the horizontal box (HBox) has attracted more and more attention. In this paper, we explore a more challenging yet label-efficient setting, namely single point-supervised OOD, and present our approach called Point2RBox. Specifically, we propose to leverage two principles: 1) Synthetic pattern knowledge combination: By sampling around each labeled point on the image, we spread the object feature to synthetic visual patterns with known boxes to provide the knowledge for box regression. 2) Transform self-supervision: With a transformed input image (e.g. scaled/rotated), the output RBoxes are trained to follow the same transformation so that the network can perceive the relative size/rotation between objects. The detector is further enhanced by a few devised techniques to cope with peripheral issues, e.g. the anchor/layer assignment as the size of the object is not available in our point supervision setting. To our best knowledge, Point2RBox is the first end-to-end solution for point-supervised OOD. In particular, our method uses a lightweight paradigm, yet it achieves a competitive performance among point-supervised alternatives, 41.05%/27.62%/80.01% on DOTA/DIOR/HRSC datasets., Comment: 10 pages, 3 figures, 5 tables, code: https://github.com/yuyi1005/point2rbox-mmrotate

Single-point annotation in oriented object detection of remote sensing scenarios is gaining increasing attention due to its cost-effectiveness. However, due to the granularity ambiguity of points, there is a significant performance gap between previous methods and those with fully supervision. In this study, we introduce P2RBox, which employs point prompt to generate rotated box (RBox) annotation for oriented object detection. P2RBox employs the SAM model to generate high-quality mask proposals. These proposals are then refined using the semantic and spatial information from annotation points. The best masks are converted into oriented boxes based on the feature directions suggested by the model. P2RBox incorporates two advanced guidance cues: Boundary Sensitive Mask guidance, which leverages semantic information, and Centrality guidance, which utilizes spatial information to reduce granularity ambiguity. This combination enhances detection capabilities significantly. To demonstrate the effectiveness of this method, enhancements based on the baseline were observed by integrating three different detectors. Furthermore, compared to the state-of-the-art point-annotated generative method PointOBB, P2RBox outperforms by about 29% mAP (62.43% vs 33.31%) on DOTA-v1.0 dataset, which provides possibilities for the practical application of point annotations.

An increasingly massive number of remote-sensing images spurs the development of extensible object detectors that can detect objects beyond training categories without costly collecting new labeled data. In this paper, we aim to develop open-vocabulary object detection (OVD) technique in aerial images that scales up object vocabulary size beyond training data. The performance of OVD greatly relies on the quality of class-agnostic region proposals and pseudo-labels for novel object categories. To simultaneously generate high-quality proposals and pseudo-labels, we propose CastDet, a CLIP-activated student-teacher open-vocabulary object Detection framework. Our end-to-end framework following the student-teacher self-learning mechanism employs the RemoteCLIP model as an extra omniscient teacher with rich knowledge. By doing so, our approach boosts not only novel object proposals but also classification. Furthermore, we devise a dynamic label queue strategy to maintain high-quality pseudo labels during batch training. We conduct extensive experiments on multiple existing aerial object detection datasets, which are set up for the OVD task. Experimental results demonstrate our CastDet achieving superior open-vocabulary detection performance, e.g., reaching 46.5% mAP on VisDroneZSD novel categories, which outperforms the state-of-the-art open-vocabulary detectors by 21.0% mAP. To our best knowledge, this is the first work to apply and develop the open-vocabulary object detection technique for aerial images. The code is available at https://github.com/lizzy8587/CastDet., Comment: Accepted by ECCV2024

Quantum Spin Hall (QSH) insulators possess distinct helical in-gap states, enabling their edge states to act as one-dimensional conducting channels when backscattering is prohibited by time-reversal symmetry. However, it remains challenging to achieve high-performance combinations of nontrivial topological QSH states with superconductivity for applications and requires understanding of the complicated underlying mechanisms. Here, our experimental observations for a novel superconducting phase in the pressurized QSH insulator Ta2Pd3Te5 is reported, and the high-pressure phase maintains its original ambient pressure lattice symmetry up to 45 GPa. Our in-situ high-pressure synchrotron X-ray diffraction, electrical transport, infrared reflectance, and Raman spectroscopy measurements, in combination with rigorous theoretical calculations, provide compelling evidence for the association between the superconducting behavior and the abnormal densified phase. The isostructural transition was found to modify the topology of the Fermi surface directly, accompanied by a fivefold amplification of the density of states at 20 GPa compared to ambient pressure, which synergistically fosters the emergence of robust superconductivity. A profound comprehension of the fascinating properties exhibited by the compressed Ta2Pd3Te5 phase is achieved, highlighting the extraordinary potential of van der Waals (vdW) QSH insulators for exploring and investigating high-performance electronic advanced devices under extreme conditions., Comment: 16pages,4figures