Search

Point cloud based 3D deep model has wide applications in many applications such as autonomous driving, house robot, and so on. Inspired by the recent prompt learning in natural language processing, this work proposes a novel Multi-view Vision-Prompt Fusion Network (MvNet) for few-shot 3D point cloud classification. MvNet investigates the possibility of leveraging the off-the-shelf 2D pre-trained models to achieve the few-shot classification, which can alleviate the over-dependence issue of the existing baseline models towards the large-scale annotated 3D point cloud data. Specifically, MvNet first encodes a 3D point cloud into multi-view image features for a number of different views. Then, a novel multi-view prompt fusion module is developed to effectively fuse information from different views to bridge the gap between 3D point cloud data and 2D pre-trained models. A set of 2D image prompts can then be derived to better describe the suitable prior knowledge for a large-scale pre-trained image model for few-shot 3D point cloud classification. Extensive experiments on ModelNet, ScanObjectNN, and ShapeNet datasets demonstrate that MvNet achieves new state-of-the-art performance for 3D few-shot point cloud image classification. The source code of this work will be available soon., 10 pages,5 figures

3D dense captioning aims to generate multiple captions localized with their associated object regions. Existing methods follow a sophisticated ``detect-then-describe'' pipeline equipped with numerous hand-crafted components. However, these hand-crafted components would yield suboptimal performance given cluttered object spatial and class distributions among different scenes. In this paper, we propose a simple-yet-effective transformer framework Vote2Cap-DETR based on recent popular \textbf{DE}tection \textbf{TR}ansformer (DETR). Compared with prior arts, our framework has several appealing advantages: 1) Without resorting to numerous hand-crafted components, our method is based on a full transformer encoder-decoder architecture with a learnable vote query driven object decoder, and a caption decoder that produces the dense captions in a set-prediction manner. 2) In contrast to the two-stage scheme, our method can perform detection and captioning in one-stage. 3) Without bells and whistles, extensive experiments on two commonly used datasets, ScanRefer and Nr3D, demonstrate that our Vote2Cap-DETR surpasses current state-of-the-arts by 11.13\% and 7.11\% in CIDEr@0.5IoU, respectively. Codes will be released soon.

Text-video retrieval contains various challenges, including biases coming from diverse sources. We highlight some of them supported by illustrations to open a discussion. Besides, we address one of the biases, frame length bias, with a simple method which brings a very incremental but promising increase. We conclude with future directions., Comment: 4 pages, CVPR 2023 Joint Ego4D&EPIC Workshop, Extended Abstract

Platelet-lymphocyte and neutrophil-lymphocyte ratio are prognostic markers for pheochromocytomas and paragangliomas:Supplemental Material

With the emergence of social media, voluminous video clips are uploaded every day, and retrieving the most relevant visual content with a language query becomes critical. Most approaches aim to learn a joint embedding space for plain textual and visual contents without adequately exploiting their intra-modality structures and inter-modality correlations. This paper proposes a novel transformer that explicitly disentangles the text and video into semantic roles of objects, spatial contexts and temporal contexts with an attention scheme to learn the intra- and inter-role correlations among the three roles to discover discriminative features for matching at different levels. The preliminary results on popular YouCook2 indicate that our approach surpasses a current state-of-the-art method, with a high margin in all metrics. It also overpasses two SOTA methods in terms of two metrics., Comment: Camera-ready for ICIP 2021

Due to the emergence of powerful computing resources and large-scale annotated datasets, deep learning has seen wide applications in our daily life. However, most current methods require extensive data collection and retraining when dealing with novel classes never seen before. On the other hand, we humans can quickly recognize new classes by looking at a few samples, which motivates the recent popularity of few-shot learning (FSL) in machine learning communities. Most current FSL approaches work on 2D image domain, however, its implication in 3D perception is relatively under-explored. Not only needs to recognize the unseen examples as in 2D domain, 3D few-shot learning is more challenging with unordered structures, high intra-class variances, and subtle inter-class differences. Moreover, different architectures and learning algorithms make it difficult to study the effectiveness of existing 2D methods when migrating to the 3D domain. In this work, for the first time, we perform systematic and extensive studies of recent 2D FSL and 3D backbone networks for benchmarking few-shot point cloud classification, and we suggest a strong baseline and learning architectures for 3D FSL. Then, we propose a novel plug-and-play component called Cross-Instance Adaptation (CIA) module, to address the high intra-class variances and subtle inter-class differences issues, which can be easily inserted into current baselines with significant performance improvement. Extensive experiments on two newly introduced benchmark datasets, ModelNet40-FS and ShapeNet70-FS, demonstrate the superiority of our proposed network for 3D FSL., Comment: Accepted by WACV 2022. arXiv admin note: text overlap with arXiv:2303.18210

In this report, we present our approach for EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge 2022. We first parse sentences into semantic roles corresponding to verbs and nouns; then utilize self-attentions to exploit semantic role contextualized video features along with textual features via triplet losses in multiple embedding spaces. Our method overpasses the strong baseline in normalized Discounted Cumulative Gain (nDCG), which is more valuable for semantic similarity. Our submission is ranked 3rd for nDCG and ranked 4th for mAP., Comment: Ranked joint 3rd place in the Multi-Instance Retrieval Challenge at EPIC@CVPR2022

Seas of videos are uploaded daily with the popularity of social channels; thus, retrieving the most related video contents with user textual queries plays a more crucial role. Most methods consider only one joint embedding space between global visual and textual features without considering the local structures of each modality. Some other approaches consider multiple embedding spaces consisting of global and local features separately, ignoring rich inter-modality correlations. We propose a novel mixture-of-expert transformer RoME that disentangles the text and the video into three levels; the roles of spatial contexts, temporal contexts, and object contexts. We utilize a transformer-based attention mechanism to fully exploit visual and text embeddings at both global and local levels with mixture-of-experts for considering inter-modalities and structures' correlations. The results indicate that our method outperforms the state-of-the-art methods on the YouCook2 and MSR-VTT datasets, given the same visual backbone without pre-training. Finally, we conducted extensive ablation studies to elucidate our design choices., Comment: Preprint, under review in TCSVT Journal

In this paper, the incremental nonlinear dynamic inversion (INDI) was applied to the design of angular rate control law to improve the robustness of the control system. Firstly, the control structure of the INDI was deduced and its robustness was analyzed. Subsequently, the angular rate controller design based on the INDI was completed, and the influence of surface damage on the aircraft was analyzed. Finally, the simulation performance of the designed angular rate controller was verified under the surface damage fault. The simulation results demonstrated that the INDI control has strong robustness, and the controller based on this method can achieve the desired control effect.

A large amount of annotated training images is critical for training accurate and robust deep network models but the collection of a large amount of annotated training images is often time-consuming and costly. Image synthesis alleviates this constraint by generating annotated training images automatically by machines which has attracted increasing interest in the recent deep learning research. We develop an innovative image synthesis technique that composes annotated training images by realistically embedding foreground objects of interest (OOI) into background images. The proposed technique consists of two key components that in principle boost the usefulness of the synthesized images in deep network training. The first is context-aware semantic coherence which ensures that the OOI are placed around semantically coherent regions within the background image. The second is harmonious appearance adaptation which ensures that the embedded OOI are agreeable to the surrounding background from both geometry alignment and appearance realism. The proposed technique has been evaluated over two related but very different computer vision challenges, namely, scene text detection and scene text recognition. Experiments over a number of public datasets demonstrate the effectiveness of our proposed image synthesis technique - the use of our synthesized images in deep network training is capable of achieving similar or even better scene text detection and scene text recognition performance as compared with using real images., Comment: This work has been merged into another project

e21044 Background: The purpose of this study was to investigate whether the combined radiomic model based on tumor-associated and margin-related (5mm) radiomic features can effectively improve prediction performance of distinguishing precancerous lesions from early stage lung adenocarcinoma. Methods: 264 patients underwent preoperative chest CT in Guangdong Provincial People’s hospital from March 1, 2015 to December 31,2019 were sorted by three cohorts. All lesions were pathologically confirmed as precancerous lesions or Stage I lung adenocarcinoma and a total of 861 analyzable radiomic features were extracted from two segmented lesions including pulmonary lesions and margins, using PyRadiomics by two senior radiologists. In training cohort, 145 patients (70%) are selected randomly from the single-nodular patients (N = 207). As for the validation cohorts, the models were validated using the resting 62 patients from single-nodular cohort and multi-nodular cohort (n = 57) respectively. Least Absolute Shrinkage and Selector Operation and Support Vector Machine-Recursive Feature Elimination were used for feature selection. ROC analysis and AUC were used to evaluate the performance of three models which were developed by multiple logistic regression on distinguishing the precancerous lesions from early stage lung adenocarcinoma. Results: Selected features from pulmonary lesions and pericarcinous tissue were developed into two independent radiomic models and a combined model. Margin-related radiomic model performs well in three validation cohorts. The AUC Brock of single-nodular cohort in training cohort was 0.912 (95% CI: 0.876-0.948), while in single-nodular validation cohort was 0.93 (95% CI: 0.862-0.966). Multi-nodular validation cohort in this model shows an AUC of 0.891 (95% CI = 0.824–0.943). Comparing combined model and tumor-associated radiomic model, it is found that the AUC of combined model was improved from 0.865 (95% CI: 0.767-0.963) to 0.94 (95% CI: 0.767-0.963) for single-nodular validation cohort. Respectively, this combined model also performs well in multi-nodular validation cohort. Conclusions: This study demonstrated the potential of margin-related radiomic features based on preoperative CT scans to distinguish precancerous lesions from early stage lung adenocarcinoma. The constructed radiomic model provided an easy-to-use, preoperative tool for surgeons to develop accurate therapeutic strategies for multi-nodular patients.

The YouTube-8M video classification challenge requires teams to classify 0.7 million videos into one or more of 4,716 classes. In this Kaggle competition, we placed in the top 3% out of 650 participants using released video and audio features. Beyond that, we extend the original competition by including text information in the classification, making this a truly multi-modal approach with vision, audio and text. The newly introduced text data is termed as YouTube-8M-Text. We present a classification framework for the joint use of text, visual and audio features, and conduct an extensive set of experiments to quantify the benefit that this additional mode brings. The inclusion of text yields state-of-the-art results, e.g. 86.7% GAP on the YouTube-8M-Text validation dataset., Comment: 8 pages, Accepted to CVPR'17 Workshop on YouTube-8M Large-Scale Video Understanding

In this paper, through the research of mass transfer mechanism of the extraction process of soluble solids (SS) in Russian olive, the kinetic model of concentration change for the SS with the solvent multiple B (mL/g), extraction temperature T (K) and extraction time t (h) during extraction process was established for the first time. Russian olive extract concentration values (%) can be calculated in different extraction process, and the extraction process can also be predicted and controlled if the extraction temperature, extraction time and a multiple of solvent are provided. Overall, the absolute errors of the kinetic model are less than ± 1% and the χ2 values are less than χ20.995 values also. Key words: Russian olive, soluble solid, extraction, mass transfer mechanism.

In recent years, China's energy consumption growth led to a sharp increase in the investment and financing scale of energy industry, and it aroused wide concern how the scale growth and structure change of energy industry investment and financing affects the development of energy industry. This article examines the impact of energy industry investment structure on the energy industry development by using time series data from 2003 to 2009, and analyses the effect of energy industry financing structure on its development by using time series data from 2004 to 2009.The results show that the growth of energy industry investment scale can improve the energy industry scale growth, but the effect for increasing its profitability is not significant,which indicates that investment efficiency is not obvious. And the effect of different financing structure on output growth and profit contribution is significant. As a result, the investment efficiency and financing structure of energy industry needs to be improved.

In this study, we increased concentration of lipopeptides by improving culture medium components and fermentation conditions, thereby laying the foundation for the industrial production of antibacterial lipopeptides. By studying optimized carbon and nitrogen sources, Plackett-Burman design, steepest ascent experiment and response surface methodology, for its medium and fermentation conditions were optimized. After the optimization, the optimized culture medium components and fermentation conditions were as follows: glucose 36.28g/L, yeast extract powder 12.77g/L,MgSO4 0.5 g/L,KCl 0.5 g/L,KH2PO41.0 g/L,FeSO4 0.15mg/L,MnSO4 5.0 mg/L,CuSO4 0.16 mg/L,inoculation quantity 4%,initial pH 6.82, temperature 37oC,time 51.69 h,rotation speed 225 r/min and loaded liquid 100 ml. Under these conditions, the yield of lipopeptides increased substantially.

The aim of this study was to screen and investigate the anti-HBV and anti-tumor activities of separated compounds from Rabdosia serra (Maxim.) Hara to lay the basis for further isolate active entity. Three kinds of extractions from Rabdosia serra using different solvents (petroleum ether, acetidin, butyl alcohol) were prepared and used to analyze their anti-HBV activity in HepG2.2.15 cells for further separation. The cytotoxicity of each extraction was tested by MTT assay, the levels of HBsAg, HBeAg and HBV DNA in supernatants from HepG2.2.15 cells were detected by ELISA and real-time quantitative polymerase chain reaction (PCR). Then, the most effective extraction was further separated, the anti-HBV activities of separated compounds were also tested by MTT and ELISA, and three compounds with highest cytotoxicity were selected to further identify their anti-tumor activities on MCF-7, BGC-823 and HepG2 cells. Acetidin extraction C2 had the most effective anti-HBV activity that was used to be further separated, it led to statistically significant reduction in HBsAg and HBeAg secretion and HBV DNA. The separation of C2 resulted in 14 compounds, A3 and A5 markedly inhibited HBsAg secretion, while A9 inhibited HBeAg secretion in a dose-dependent manner with higher TI comparing with C2. A6, A7, A11 had different anti-tumor activity against different tumor cells. These data showed that the extraction and their separated effective compounds had strong inhibitory effect on HBV replication so as to have anti-HBV activity, and further separation and purification could enhance anti-HBV activity. Meanwhile, some compounds have high cytotoxicities on different tumor cells. Our study could provide a theoretical basis for the next clinical use and the development of potential and efficient drugs for HBV and tumor therapy from Rabdosia serra.