Search

In this paper, we perform robot manipulation activities in real-world environments with language contexts by integrating a compact referring image segmentation model into the robot's perception module. First, we propose CLIPU$^2$Net, a lightweight referring image segmentation model designed for fine-grain boundary and structure segmentation from language expressions. Then, we deploy the model in an eye-in-hand visual servoing system to enact robot control in the real world. The key to our system is the representation of salient visual information as geometric constraints, linking the robot's visual perception to actionable commands. Experimental results on 46 real-world robot manipulation tasks demonstrate that our method outperforms traditional visual servoing methods relying on labor-intensive feature annotations, excels in fine-grain referring image segmentation with a compact decoder size of 6.6 MB, and supports robot control across diverse contexts.

We show Kawamata's effective nonvanishing conjecture (also known as the Ambro--Kawamata nonvanishing conjecture) holds for quasismooth weighted complete intersections of codimension $2$. Namely, for a quasismooth weighted complete intersection $X$ of codimension $2$ and an ample Cartier divisor $H$ on $X$ such that $H-K_X$ is ample, the linear system $|H|$ is nonempty., Comment: 13 pages, to appear in Kyoto J. Math

Let $X$ be a normal projective variety of dimension $d$ over an algebraically closed field and $f$ an automorphism of $X$. Suppose that the pullback $f^*|_{\mathsf{N}^1(X)_\mathbf{R}}$ of $f$ on the real N\'eron--Severi space $\mathsf{N}^1(X)_\mathbf{R}$ is unipotent and denote the index of the eigenvalue $1$ by $k+1$. We prove an upper bound for the polynomial volume growth $\mathrm{plov}(f)$ of $f$ as follows: \[ \mathrm{plov}(f) \le (k/2 + 1)d. \] This inequality is optimal in certain cases. Furthermore, we show that $k\le 2(d-1)$, extending a result of Dinh--Lin--Oguiso--Zhang for compact K\"ahler manifolds to arbitrary characteristic. Combining these two inequalities together, we obtain an optimal inequality that \[ \mathrm{plov}(f) \le d^2, \] which affirmatively answers questions of Cantat--Paris-Romaskevich and Lin--Oguiso--Zhang., Comment: 26 pages, comments are welcome

We prove that the canonical model of a $3$-fold of general type with geometric genus $2$ and with minimal canonical volume $\frac{1}{3}$ must be a hypersurface of degree $16$ in $\mathbb{P}(1,1,2,3,8)$, which gives an explicit description of its canonical ring. This implies that the coarse moduli space $\mathcal{M}_{\frac{1}{3}, 2}$, parametrizing all canonical $3$-folds with canonical volume $\frac{1}{3}$ and geometric genus $2$, is an irreducible variety of dimension $189$. Parallel studies show that $\mathcal{M}_{1, 3}$ is irreducible as well and is of dimension $236$, and that $\mathcal{M}_{2, 4}$ is irreducible and is of dimension $270$. As being conceived, every member in these 3 families is simply connected. Additionally, our method yields the expected Noether inequality $\textrm{Vol}>\frac{4}{3}p_g-\frac{10}{3}$ for $3$-folds of general type with $p_g=5$., Comment: Comments are very welcome

PLATO (PLAnetary Transits and Oscillations of stars) is ESA's M3 mission designed to detect and characterise extrasolar planets and perform asteroseismic monitoring of a large number of stars. PLATO will detect small planets (down to <2 R_(Earth)) around bright stars (<11 mag), including terrestrial planets in the habitable zone of solar-like stars. With the complement of radial velocity observations from the ground, planets will be characterised for their radius, mass, and age with high accuracy (5 %, 10 %, 10 % for an Earth-Sun combination respectively). PLATO will provide us with a large-scale catalogue of well-characterised small planets up to intermediate orbital periods, relevant for a meaningful comparison to planet formation theories and to better understand planet evolution. It will make possible comparative exoplanetology to place our Solar System planets in a broader context. In parallel, PLATO will study (host) stars using asteroseismology, allowing us to determine the stellar properties with high accuracy, substantially enhancing our knowledge of stellar structure and evolution. The payload instrument consists of 26 cameras with 12cm aperture each. For at least four years, the mission will perform high-precision photometric measurements. Here we review the science objectives, present PLATO's target samples and fields, provide an overview of expected core science performance as well as a description of the instrument and the mission profile at the beginning of the serial production of the flight cameras. PLATO is scheduled for a launch date end 2026. This overview therefore provides a summary of the mission to the community in preparation of the upcoming operational phases.

Modeling and analyzing long sequences of text is an essential task for Natural Language Processing. Success in capturing long text dynamics using neural language models will facilitate many downstream tasks such as coherence evaluation, text generation, machine translation and so on. This paper presents a novel approach to model sequences through a stochastic process. We introduce a likelihood-based training objective for the text encoder and design a more thorough measurement (score) for long text evaluation compared to the previous approach. The proposed training objective effectively preserves the sequence coherence, while the new score comprehensively captures both temporal and spatial dependencies. Theoretical properties of our new score show its advantages in sequence evaluation. Experimental results show superior performance in various sequence evaluation tasks, including global and local discrimination within and between documents of different lengths. We also demonstrate the encoder achieves competitive results on discriminating human and AI written text.

The user base of short video apps has experienced unprecedented growth in recent years, resulting in a significant demand for video content analysis. In particular, text-video retrieval, which aims to find the top matching videos given text descriptions from a vast video corpus, is an essential function, the primary challenge of which is to bridge the modality gap. Nevertheless, most existing approaches treat texts merely as discrete tokens and neglect their syntax structures. Moreover, the abundant spatial and temporal clues in videos are often underutilized due to the lack of interaction with text. To address these issues, we argue that using texts as guidance to focus on relevant temporal frames and spatial regions within videos is beneficial. In this paper, we propose a novel Syntax-Hierarchy-Enhanced text-video retrieval method (SHE-Net) that exploits the inherent semantic and syntax hierarchy of texts to bridge the modality gap from two perspectives. First, to facilitate a more fine-grained integration of visual content, we employ the text syntax hierarchy, which reveals the grammatical structure of text descriptions, to guide the visual representations. Second, to further enhance the multi-modal interaction and alignment, we also utilize the syntax hierarchy to guide the similarity calculation. We evaluated our method on four public text-video retrieval datasets of MSR-VTT, MSVD, DiDeMo, and ActivityNet. The experimental results and ablation studies confirm the advantages of our proposed method.

We investigated the behavior of fractional quantum Hall (FQH) states in a two-dimensional electron system with layer thickness and an in-plane magnetic field. Our comparisons across various filling factors within the first Landau level revealed a crucial observation. A slight in-plane magnetic field specifically enhances the nematic order of the $\nu = 7/3$ FQH state. For this particular filling, through calculating the energy gap, the Ising nematic order parameter, the pair-correlation function, and the static structure factor, we observed that as the in-plane magnetic field increases, the system first enters into an anisotropic FQH phase without closing the spectrum gap, then the FQH nematic (FQHN) phase after neutral gap closing. The system eventually enters a gapless one-dimensional charge density wave (CDW) phase for a large in-plane field. We thus provide a full phase diagram of the $\nu = 7/3$ state in a tilted magnetic field, demonstrating the existence of the FQHN, which aligns with recent resonant inelastic light scattering (RILS) experimental observations., Comment: 9 pages, 11 figures

With the explosive growth of video data in real-world applications, a comprehensive representation of videos becomes increasingly important. In this paper, we address the problem of video scene recognition, whose goal is to learn a high-level video representation to classify scenes in videos. Due to the diversity and complexity of video contents in realistic scenarios, this task remains a challenge. Most existing works identify scenes for videos only from visual or textual information in a temporal perspective, ignoring the valuable information hidden in single frames, while several earlier studies only recognize scenes for separate images in a non-temporal perspective. We argue that these two perspectives are both meaningful for this task and complementary to each other, meanwhile, externally introduced knowledge can also promote the comprehension of videos. We propose a novel two-stream framework to model video representations from multiple perspectives, i.e. temporal and non-temporal perspectives, and integrate the two perspectives in an end-to-end manner by self-distillation. Besides, we design a knowledge-enhanced feature fusion and label prediction method that contributes to naturally introducing knowledge into the task of video scene recognition. Experiments conducted on a real-world dataset demonstrate the effectiveness of our proposed method.

Measuring the coherence of text is a vital aspect of evaluating the quality of written content. Recent advancements in neural coherence modeling have demonstrated their efficacy in capturing entity coreference and discourse relations, thereby enhancing coherence evaluation. However, many existing methods heavily depend on static embeddings or focus narrowly on nearby context, constraining their capacity to measure the overarching coherence of long texts. In this paper, we posit that coherent texts inherently manifest a sequential and cohesive interplay among sentences, effectively conveying the central theme, purpose, or standpoint. To explore this abstract relationship, we introduce the "BBScore," a novel reference-free metric grounded in Brownian bridge theory for assessing text coherence. Our findings showcase that when synergized with a simple additional classification component, this metric attains a performance level comparable to state-of-the-art techniques on standard artificial discrimination tasks. We also establish in downstream tasks that this metric effectively differentiates between human-written documents and text generated by large language models under a specific domain. Furthermore, we illustrate the efficacy of this approach in detecting written styles attributed to diverse large language models, underscoring its potential for generalizability. In summary, we present a novel Brownian bridge coherence metric capable of measuring both local and global text coherence, while circumventing the need for end-to-end model training. This flexibility allows for its application in various downstream tasks., Comment: Accepted to the 38th Annual AAAI Conference on Artificial Intelligence (AAAI-24)

The Segment Anything Model (SAM) has recently emerged as a groundbreaking foundation model for prompt-driven image segmentation tasks. However, both the original SAM and its medical variants require slice-by-slice manual prompting of target structures, which directly increase the burden for applications. Despite attempts of auto-prompting to turn SAM into a fully automatic manner, it still exhibits subpar performance and lacks of reliability especially in the field of medical imaging. In this paper, we propose UR-SAM, an uncertainty rectified SAM framework to enhance the reliability for auto-prompting medical image segmentation. Building upon a localization framework for automatic prompt generation, our method incorporates a prompt augmentation module to obtain a series of input prompts for SAM for uncertainty estimation and an uncertainty-based rectification module to further utilize the distribution of estimated uncertainty to improve the segmentation performance. Extensive experiments on two public 3D medical datasets covering the segmentation of 35 organs demonstrate that without supplementary training or fine-tuning, our method further improves the segmentation performance with up to 10.7 % and 13.8 % in dice similarity coefficient, demonstrating efficiency and broad capabilities for medical image segmentation without manual prompting.

In this paper, we propose a framework of building knowledgeable robot control in the scope of smart human-robot interaction, by empowering a basic uncalibrated visual servoing controller with contextual knowledge through the joint usage of event knowledge graphs (EKGs) and large-scale pretrained vision-language models (VLMs). The framework is expanded in twofold: first, we interpret low-level image geometry as high-level concepts, allowing us to prompt VLMs and to select geometric features of points and lines for motor control skills; then, we create an event knowledge graph (EKG) to conceptualize a robot manipulation task of interest, where the main body of the EKG is characterized by an executable behavior tree, and the leaves by semantic concepts relevant to the manipulation context. We demonstrate, in an uncalibrated environment with real robot trials, that our method lowers the reliance of human annotation during task interfacing, allows the robot to perform activities of daily living more easily by treating low-level geometric-based motor control skills as high-level concepts, and is beneficial in building cognitive thinking for smart robot applications.

With the explosive growth of web videos in recent years, large-scale Content-Based Video Retrieval (CBVR) becomes increasingly essential in video filtering, recommendation, and copyright protection. Segment-level CBVR (S-CBVR) locates the start and end time of similar segments in finer granularity, which is beneficial for user browsing efficiency and infringement detection especially in long video scenarios. The challenge of S-CBVR task is how to achieve high temporal alignment accuracy with efficient computation and low storage consumption. In this paper, we propose a Segment Similarity and Alignment Network (SSAN) in dealing with the challenge which is firstly trained end-to-end in S-CBVR. SSAN is based on two newly proposed modules in video retrieval: (1) An efficient Self-supervised Keyframe Extraction (SKE) module to reduce redundant frame features, (2) A robust Similarity Pattern Detection (SPD) module for temporal alignment. In comparison with uniform frame extraction, SKE not only saves feature storage and search time, but also introduces comparable accuracy and limited extra computation time. In terms of temporal alignment, SPD localizes similar segments with higher accuracy and efficiency than existing deep learning methods. Furthermore, we jointly train SSAN with SKE and SPD and achieve an end-to-end improvement. Meanwhile, the two key modules SKE and SPD can also be effectively inserted into other video retrieval pipelines and gain considerable performance improvements. Experimental results on public datasets show that SSAN can obtain higher alignment accuracy while saving storage and online query computational cost compared to existing methods., Comment: Accepted by ACM MM 2021

In recent years, the explosion of web videos makes text-video retrieval increasingly essential and popular for video filtering, recommendation, and search. Text-video retrieval aims to rank relevant text/video higher than irrelevant ones. The core of this task is to precisely measure the cross-modal similarity between texts and videos. Recently, contrastive learning methods have shown promising results for text-video retrieval, most of which focus on the construction of positive and negative pairs to learn text and video representations. Nevertheless, they do not pay enough attention to hard negative pairs and lack the ability to model different levels of semantic similarity. To address these two issues, this paper improves contrastive learning using two novel techniques. First, to exploit hard examples for robust discriminative power, we propose a novel Dual-Modal Attention-Enhanced Module (DMAE) to mine hard negative pairs from textual and visual clues. By further introducing a Negative-aware InfoNCE (NegNCE) loss, we are able to adaptively identify all these hard negatives and explicitly highlight their impacts in the training loss. Second, our work argues that triplet samples can better model fine-grained semantic similarity compared to pairwise samples. We thereby present a new Triplet Partial Margin Contrastive Learning (TPM-CL) module to construct partial order triplet samples by automatically generating fine-grained hard negatives for matched text-video pairs. The proposed TPM-CL designs an adaptive token masking strategy with cross-modal interaction to model subtle semantic differences. Extensive experiments demonstrate that the proposed approach outperforms existing methods on four widely-used text-video retrieval datasets, including MSR-VTT, MSVD, DiDeMo and ActivityNet., Comment: Accepted by ACM MM 2023

The classical human-robot interface in uncalibrated image-based visual servoing (UIBVS) relies on either human annotations or semantic segmentation with categorical labels. Both methods fail to match natural human communication and convey rich semantics in manipulation tasks as effectively as natural language expressions. In this paper, we tackle this problem by using referring expression segmentation, which is a prompt-based approach, to provide more in-depth information for robot perception. To generate high-quality segmentation predictions from referring expressions, we propose CLIPUNetr - a new CLIP-driven referring expression segmentation network. CLIPUNetr leverages CLIP's strong vision-language representations to segment regions from referring expressions, while utilizing its ``U-shaped'' encoder-decoder architecture to generate predictions with sharper boundaries and finer structures. Furthermore, we propose a new pipeline to integrate CLIPUNetr into UIBVS and apply it to control robots in real-world environments. In experiments, our method improves boundary and structure measurements by an average of 120% and can successfully assist real-world UIBVS control in an unstructured manipulation environment.

Social networks exhibit a complex graph-like structure due to the uncertainty surrounding potential collaborations among participants. Machine learning algorithms possess generic outstanding performance in multiple real-world prediction tasks. However, whether machine learning algorithms outperform specific algorithms designed for graph link prediction remains unknown to us. To address this issue, the Adamic-Adar Index (AAI), Jaccard Coefficient (JC) and common neighbour centrality (CNC) as representatives of graph-specific algorithms were applied to predict potential collaborations, utilizing data from volunteer activities during the Covid-19 pandemic in Shenzhen city, along with the classical machine learning algorithms such as random forest, support vector machine, and gradient boosting as single predictors and components of ensemble learning. This paper introduces that the AAI algorithm outperformed the traditional JC and CNC, and other machine learning algorithms in analyzing graph node attributes for this task.

Recently, multi-expert methods have led to significant improvements in long-tail recognition (LTR). We summarize two aspects that need further enhancement to contribute to LTR boosting: (1) More diverse experts; (2) Lower model variance. However, the previous methods didn't handle them well. To this end, we propose More Diverse experts with Consistency Self-distillation (MDCS) to bridge the gap left by earlier methods. Our MDCS approach consists of two core components: Diversity Loss (DL) and Consistency Self-distillation (CS). In detail, DL promotes diversity among experts by controlling their focus on different categories. To reduce the model variance, we employ KL divergence to distill the richer knowledge of weakly augmented instances for the experts' self-distillation. In particular, we design Confident Instance Sampling (CIS) to select the correctly classified instances for CS to avoid biased/noisy knowledge. In the analysis and ablation study, we demonstrate that our method compared with previous work can effectively increase the diversity of experts, significantly reduce the variance of the model, and improve recognition accuracy. Moreover, the roles of our DL and CS are mutually reinforcing and coupled: the diversity of experts benefits from the CS, and the CS cannot achieve remarkable results without the DL. Experiments show our MDCS outperforms the state-of-the-art by 1% $\sim$ 2% on five popular long-tailed benchmarks, including CIFAR10-LT, CIFAR100-LT, ImageNet-LT, Places-LT, and iNaturalist 2018. The code is available at https://github.com/fistyee/MDCS., Comment: ICCV2023 Accept. 13 pages

We prove that the automorphism group of a compact hyperk\"ahler manifold of dimension 4 acts faithfully on the cohomology ring., Comment: 10 pages, final version to appear in Math. Res. Lett

When main-sequence stars expand into red giants, they are expected to engulf close-in planets. Until now, the absence of planets with short orbital periods around post-expansion, core-helium-burning red giants has been interpreted as evidence that short-period planets around Sun-like stars do not survive the giant expansion phase of their host stars. Here we present the discovery that the giant planet 8 Ursae Minoris b orbits a core-helium-burning red giant. At a distance of only 0.5 au from its host star, the planet would have been engulfed by its host star, which is predicted by standard single-star evolution to have previously expanded to a radius of 0.7 au. Given the brief lifetime of helium-burning giants, the nearly circular orbit of the planet is challenging to reconcile with scenarios in which the planet survives by having a distant orbit initially. Instead, the planet may have avoided engulfment through a stellar merger that either altered the evolution of the host star or produced 8 Ursae Minoris b as a second-generation planet. This system shows that core-helium-burning red giants can harbour close planets and provides evidence for the role of non-canonical stellar evolution in the extended survival of late-stage exoplanetary systems., Comment: Published in Nature on 28 June 2023. In press

Since the invention of modern CT systems, metal artifacts have been a persistent problem. Due to increased scattering, amplified noise, and insufficient data collection, it is more difficult to suppress metal artifacts in cone-beam CT, limiting its use in human- and robot-assisted spine surgeries where metallic guidewires and screws are commonly used. In this paper, we demonstrate that conventional image-domain segmentation-based MAR methods are unable to eliminate metal artifacts for intraoperative CBCT images with guidewires. To solve this problem, we present a fine-grained projection-domain segmentation-based MAR method termed PDS-MAR, in which metal traces are augmented and segmented in the projection domain before being inpainted using triangular interpolation. In addition, a metal reconstruction phase is proposed to restore metal areas in the image domain. The digital phantom study and real CBCT data study demonstrate that the proposed algorithm achieves significantly better artifact suppression than other comparing methods and has the potential to advance the use of intraoperative CBCT imaging in clinical spine surgeries., Comment: 19 Pages

We answer an open problem of the first author and Zhang (see Open Problem 6.4 (3) in Math. Z. 258 (2008), 565-585) and prove that, for any nonsingular projective $3$-fold of general type with the geometric genus greater than 201, the bicanonical system is not composed of any pencil of surfaces., Comment: (To appear in) Acta Math Sinica (English Series), 8 pages

Image retrieval plays an important role in the Internet world. Usually, the core parts of mainstream visual retrieval systems include an online service of the embedding model and a large-scale vector database. For traditional model upgrades, the old model will not be replaced by the new one until the embeddings of all the images in the database are re-computed by the new model, which takes days or weeks for a large amount of data. Recently, backward-compatible training (BCT) enables the new model to be immediately deployed online by making the new embeddings directly comparable to the old ones. For BCT, improving the compatibility of two models with less negative impact on retrieval performance is the key challenge. In this paper, we introduce AdvBCT, an Adversarial Backward-Compatible Training method with an elastic boundary constraint that takes both compatibility and discrimination into consideration. We first employ adversarial learning to minimize the distribution disparity between embeddings of the new model and the old model. Meanwhile, we add an elastic boundary constraint during training to improve compatibility and discrimination efficiently. Extensive experiments on GLDv2, Revisited Oxford (ROxford), and Revisited Paris (RParis) demonstrate that our method outperforms other BCT methods on both compatibility and discrimination. The implementation of AdvBCT will be publicly available at https://github.com/Ashespt/AdvBCT., Comment: accepted by CVPR 2023

For a real number $0<\epsilon<1/3$, we show that the anti-canonical volume of an $\epsilon$-klt Fano $3$-fold is at most $3200/\epsilon^4$ and the order $O(1/\epsilon^4)$ is sharp., Comment: 15 pages, comments are welcome

Asteroseismology is playing an increasingly important role in the characterization of red-giant host stars and their planetary systems. Here, we conduct detailed asteroseismic modeling of the evolved red-giant branch (RGB) hosts KOI-3886 and $\iota$ Draconis, making use of end-of-mission Kepler (KOI-3886) and multi-sector TESS ($\iota$ Draconis) time-series photometry. We also model the benchmark star KIC 8410637, a member of an eclipsing binary, thus providing a direct test to the seismic determination. We test the impact of adopting different sets of observed modes as seismic constraints. Inclusion of $\ell=1$ and 2 modes improves the precision on the stellar parameters, albeit marginally, compared to adopting radial modes alone, with $1.9$-$3.0\%$ (radius), $5$-$9\%$ (mass), and $19$-$25\%$ (age) reached when using all p-dominated modes as constraints. Given the very small spacing of adjacent dipole mixed modes in evolved RGB stars, the sparse set of observed g-dominated modes is not able to provide extra constraints, further leading to highly multimodal posteriors. Access to multi-year time-series photometry does not improve matters, with detailed modeling of evolved RGB stars based on (lower-resolution) TESS data sets attaining a precision commensurate with that based on end-of-mission Kepler data. Furthermore, we test the impact of varying the atmospheric boundary condition in our stellar models. We find mass and radius estimates to be insensitive to the description of the near-surface layers, at the expense of substantially changing both the near-surface structure of the best-fitting models and the values of associated parameters like the initial helium abundance, $Y_{\rm i}$. Attempts to measure $Y_{\rm i}$ from seismic modeling of red giants may thus be systematically dependent on the choice of atmospheric physics., Comment: Accepted for publication in The Astronomical Journal (AJ)