Search

Modeling and producing lifelike clothed human images has attracted researchers' attention from different areas for decades, with the complexity from highly articulated and structured content. Rendering algorithms decompose and simulate the imaging process of a camera, while are limited by the accuracy of modeled variables and the efficiency of computation. Generative models can produce impressively vivid human images, however still lacking in controllability and editability. This paper studies photorealism enhancement of rendered images, leveraging generative power from diffusion models on the controlled basis of rendering. We introduce a novel framework to translate rendered images into their realistic counterparts, which consists of two stages: Domain Knowledge Injection (DKI) and Realistic Image Generation (RIG). In DKI, we adopt positive (real) domain finetuning and negative (rendered) domain embedding to inject knowledge into a pretrained Text-to-image (T2I) diffusion model. In RIG, we generate the realistic image corresponding to the input rendered image, with a Texture-preserving Attention Control (TAC) to preserve fine-grained clothing textures, exploiting the decoupled features encoded in the UNet structure. Additionally, we introduce SynFashion dataset, featuring high-quality digital clothing images with diverse textures. Extensive experimental results demonstrate the superiority and effectiveness of our method in rendered-to-real image translation., Comment: Accepted by NeurIPS 2024

LiDAR-based 3D object detection is a critical technology for the development of autonomous driving and robotics. However, the high cost of data annotation limits its advancement. We propose a novel and effective active learning (AL) method called Distribution Discrepancy and Feature Heterogeneity (DDFH), which simultaneously considers geometric features and model embeddings, assessing information from both the instance-level and frame-level perspectives. Distribution Discrepancy evaluates the difference and novelty of instances within the unlabeled and labeled distributions, enabling the model to learn efficiently with limited data. Feature Heterogeneity ensures the heterogeneity of intra-frame instance features, maintaining feature diversity while avoiding redundant or similar instances, thus minimizing annotation costs. Finally, multiple indicators are efficiently aggregated using Quantile Transform, providing a unified measure of informativeness. Extensive experiments demonstrate that DDFH outperforms the current state-of-the-art (SOTA) methods on the KITTI and Waymo datasets, effectively reducing the bounding box annotation cost by 56.3% and showing robustness when working with both one-stage and two-stage models., Comment: Accepted to CoRL 2024

We propose a novel end-to-end document understanding model called SeRum (SElective Region Understanding Model) for extracting meaningful information from document images, including document analysis, retrieval, and office automation. Unlike state-of-the-art approaches that rely on multi-stage technical schemes and are computationally expensive, SeRum converts document image understanding and recognition tasks into a local decoding process of the visual tokens of interest, using a content-aware token merge module. This mechanism enables the model to pay more attention to regions of interest generated by the query decoder, improving the model's effectiveness and speeding up the decoding speed of the generative scheme. We also designed several pre-training tasks to enhance the understanding and local awareness of the model. Experimental results demonstrate that SeRum achieves state-of-the-art performance on document understanding tasks and competitive results on text spotting tasks. SeRum represents a substantial advancement towards enabling efficient and effective end-to-end document understanding., Comment: Accepted to ICCV 2023 main conference

Structured text extraction is one of the most valuable and challenging application directions in the field of Document AI. However, the scenarios of past benchmarks are limited, and the corresponding evaluation protocols usually focus on the submodules of the structured text extraction scheme. In order to eliminate these problems, we organized the ICDAR 2023 competition on Structured text extraction from Visually-Rich Document images (SVRD). We set up two tracks for SVRD including Track 1: HUST-CELL and Track 2: Baidu-FEST, where HUST-CELL aims to evaluate the end-to-end performance of Complex Entity Linking and Labeling, and Baidu-FEST focuses on evaluating the performance and generalization of Zero-shot / Few-shot Structured Text extraction from an end-to-end perspective. Compared to the current document benchmarks, our two tracks of competition benchmark enriches the scenarios greatly and contains more than 50 types of visually-rich document images (mainly from the actual enterprise applications). The competition opened on 30th December, 2022 and closed on 24th March, 2023. There are 35 participants and 91 valid submissions received for Track 1, and 15 participants and 26 valid submissions received for Track 2. In this report we will presents the motivation, competition datasets, task definition, evaluation protocol, and submission summaries. According to the performance of the submissions, we believe there is still a large gap on the expected information extraction performance for complex and zero-shot scenarios. It is hoped that this competition will attract many researchers in the field of CV and NLP, and bring some new thoughts to the field of Document AI., Comment: ICDAR 2023 Competition on SVRD report (To be appear in ICDAR 2023)

SUPERB was proposed to evaluate the generalizability of self-supervised learning (SSL) speech models across various tasks. However, it incurs high computational costs due to the large datasets and diverse tasks. In this paper, we introduce MiniSUPERB, a lightweight benchmark that efficiently evaluates SSL speech models with comparable results to SUPERB but lower computational costs significantly. We carefully select representative tasks, sample datasets, and extract model representations offline. Our approach achieves a Spearman's rank correlation of 0.954 and 0.982 with SUPERB Paper and SUPERB Challenge, respectively. Additionally, we reduce the computational cost by 97% in terms of Multiply-ACcumulate operations (MACs). Furthermore, we evaluate SSL speech models in few-shot scenarios and observe significant variations in their performance. To our knowledge, this is the first study to examine both the computational cost of the model itself and the cost of evaluating it on a benchmark., Comment: Accepted to IEEE ASRU 2023

In this paper, we propose a novel method for highly efficient follicular segmentation of thyroid cytopathological WSIs. Firstly, we propose a hybrid segmentation architecture, which integrates a classifier into Deeplab V3 by adding a branch. A large amount of the WSI segmentation time is saved by skipping the irrelevant areas using the classification branch. Secondly, we merge the low scale fine features into the original atrous spatial pyramid pooling (ASPP) in Deeplab V3 to accurately represent the details in cytopathological images. Thirdly, our hybrid model is trained by a criterion-oriented adaptive loss function, which leads the model converging much faster. Experimental results on a collection of thyroid patches demonstrate that the proposed model reaches 80.9% on the segmentation accuracy. Besides, 93% time is reduced for the WSI segmentation by using our proposed method, and the WSI-level accuracy achieves 53.4%.

Computing gravity field of a mass body is a core routine to image anomalous density structures in the Earth. In this study, we report the existence of analytical routines to accurately compute the gravity potential and gravity field of a general polyhedral mass body. The density contrasts in the polyhedral body can be general polynomial functions up to arbitrary non-negative orders and also can vary in both horizontal and vertical directions. The newly derived analytical expressions of gravity fields are also singularity-free which means that observation sites can have arbitrary geometric relationships with polyhedral mass bodies. One synthetic prismatic body with different density contrasts is used to verify the accuracies of our new closed-form solutions. Excellent agreements are obtained among our solutions and other published solutions. Our work is the first-of-its kind to completely answer the fundamental question on existence of analytic solutions of gravitational field for general mass bodies. It may put an end of searching closed-form solutions in gravity surveying., Comment: 16 pages, 2 figures and one long appendix

We consider bosonic atoms loaded into optical lattices with cavity-mediated infinite-range interactions. Competing short- and global-range interactions cultivates a rich phase diagram. With a systematic field-theoretical perspective, we present an $\emph{analytical}$ construction of global ground-state phase diagram. We find that the infinite-range interaction enhances the fluctuation of the number density. In the strong coupling regime, we find four branches of elementary excitations with two being "partilce-like" and two being "hole-like", and that the excitation gap becomes soft at the phase boundary between compressible phases and incompressible phases. We derive an effective theory describing compressible superfluid and supersolid states. To complement this perturbative study, we construct a self-consistent mean-field theory and find numerical results consistent with our theoretical analysis. We map out the phase diagram and find that a charge density wave may undergo a structure phase transition to a different charge density wave before it finally enters into the supersolid phase driven by increasing the hopping amplitude., Comment: 7 pages, 7 figures

The contributions of vestibular nerve afferents and central vestibular pathways to the angular (AVOR) and linear (LVOR) vestibulo-ocular reflex were studied in squirrel monkeys during fixation of near and far targets. Irregular vestibular afferents did not appear to be necessary for the LVOR, since when they were selectively silenced with galvanic currents the LVOR was essentially unaffected during both far- and near-target viewing. The linear translation signals generated by secondary AVOR neurons in the vestibular nuclei were, on average, in phase with head velocity, inversely related to viewing distance, and were nearly as strong as AVOR-related signals. We suggest that spatial-temporal transformation of linear head translation signals to angular eye velocity commands is accomplished primarily by the addition of viewing distance multiplied, centrally integrated, otolith regular afferent signals to angular VOR pathways.