Search

This paper proposes a new generative adversarial network for pose transfer, i.e., transferring the pose of a given person to a target pose. We design a progressive generator which comprises a sequence of transfer blocks. Each block performs an intermediate transfer step by modeling the relationship between the condition and the target poses with attention mechanism. Two types of blocks are introduced, namely Pose-Attentional Transfer Block (PATB) and Aligned Pose-Attentional Transfer Bloc ~(APATB). Compared with previous works, our model generates more photorealistic person images that retain better appearance consistency and shape consistency compared with input images. We verify the efficacy of the model on the Market-1501 and DeepFashion datasets, using quantitative and qualitative measures. Furthermore, we show that our method can be used for data augmentation for the person re-identification task, alleviating the issue of data insufficiency. Code and pretrained models are available at https://github.com/tengteng95/Pose-Transfer.git., Accepted by TPAMI. An extension of the conference version. Conference version at arXiv:1904.03349

Scene text detection and recognition is a universal text recognition technology, which has become a hot research topic in the field of computer vision and document analysis in recent years. It is widely applied in geographical positioning, license plate recognition, and driverless applications. Compared to traditional document text detection and recognition, scene text varies more dramatically in font, color, scale, layout, and background. Owing to its excellent performance, deep learning has been widely adopted in this field. In this paper, we mainly review our representative studies based on deep learning in this field and describe the future research trends in this field.

Chinese scene text reading is one of the most challenging problems in computer vision and has attracted great interest. Different from English text, Chinese has more than 6000 commonly used characters and Chinese characters can be arranged in various layouts with numerous fonts. The Chinese signboards in street view are a good choice for Chinese scene text images since they have different backgrounds, fonts and layouts. We organized a competition called ICDAR2019-ReCTS, which mainly focuses on reading Chinese text on signboard. This report presents the final results of the competition. A large-scale dataset of 25,000 annotated signboard images, in which all the text lines and characters are annotated with locations and transcriptions, were released. Four tasks, namely character recognition, text line recognition, text line detection and end-to-end recognition were set up. Besides, considering the Chinese text ambiguity issue, we proposed a multi ground truth (multi-GT) evaluation method to make evaluation fairer. The competition started on March 1, 2019 and ended on April 30, 2019. 262 submissions from 46 teams are received. Most of the participants come from universities, research institutes, and tech companies in China. There are also some participants from the United States, Australia, Singapore, and Korea. 21 teams submit results for Task 1, 23 teams submit results for Task 2, 24 teams submit results for Task 3, and 13 teams submit results for Task 4. The official website for the competition is http://rrc.cvc.uab.es/?ch=12.

This paper proposes a new generative adversarial network for pose transfer, i.e., transferring the pose of a given person to a target pose. The generator of the network comprises a sequence of Pose-Attentional Transfer Blocks that each transfers certain regions it attends to, generating the person image progressively. Compared with those in previous works, our generated person images possess better appearance consistency and shape consistency with the input images, thus significantly more realistic-looking. The efficacy and efficiency of the proposed network are validated both qualitatively and quantitatively on Market-1501 and DeepFashion. Furthermore, the proposed architecture can generate training images for person re-identification, alleviating data insufficiency. Codes and models are available at: https://github.com/tengteng95/Pose-Transfer.git., Comment: To appear in CVPR 2019, oral presentation (21 pages, 15 figures including the supplementary materials)

Despite the great success of convolutional neural networks (CNNs) for the image classification task on data sets such as Cifar and ImageNet, CNN’s representation power is still somewhat limited in dealing with images that have a large variation in size and clutter, where Fisher vector (FV) has shown to be an effective encoding strategy. FV encodes an image by aggregating local descriptors with a universal generative Gaussian mixture model (GMM). FV, however, has limited learning capability and its parameters are mostly fixed after constructing the codebook. To combine together the best of the two worlds, we propose in this brief a neural network structure with FV layer being part of an end-to-end trainable system that is differentiable; we name our network FisherNet that is learnable using back propagation. Our proposed FisherNet combines CNN training and FV encoding in a single end-to-end structure. We observe a clear advantage of FisherNet over plain CNN and standard FV in terms of both classification accuracy and computational efficiency on the challenging PASCAL visual object classes object classification and emotion image classification tasks.

Scene text detection is an important step of scene text recognition system and also a challenging problem. Different from general object detection, the main challenges of scene text detection lie on arbitrary orientations, small sizes, and significantly variant aspect ratios of text in natural images. In this paper, we present an end-to-end trainable fast scene text detector, named TextBoxes++, which detects arbitrary-oriented scene text with both high accuracy and efficiency in a single network forward pass. No post-processing other than an efficient non-maximum suppression is involved. We have evaluated the proposed TextBoxes++ on four public datasets. In all experiments, TextBoxes++ outperforms competing methods in terms of text localization accuracy and runtime. More specifically, TextBoxes++ achieves an f-measure of 0.817 at 11.6fps for 1024*1024 ICDAR 2015 Incidental text images, and an f-measure of 0.5591 at 19.8fps for 768*768 COCO-Text images. Furthermore, combined with a text recognizer, TextBoxes++ significantly outperforms the state-of-the-art approaches for word spotting and end-to-end text recognition tasks on popular benchmarks. Code is available at: https://github.com/MhLiao/TextBoxes_plusplus, Comment: 15 pages

A challenging aspect of scene text recognition is to handle text with distortions or irregular layout. In particular, perspective text and curved text are common in natural scenes and are difficult to recognize. In this work, we introduce ASTER, an end-to-end neural network model that comprises a rectification network and a recognition network. The rectification network adaptively transforms an input image into a new one, rectifying the text in it. It is powered by a flexible Thin-Plate Spline transformation which handles a variety of text irregularities and is trained without human annotations. The recognition network is an attentional sequence-to-sequence model that predicts a character sequence directly from the rectified image. The whole model is trained end to end, requiring only images and their groundtruth text. Through extensive experiments, we verify the effectiveness of the rectification and demonstrate the state-of-the-art recognition performance of ASTER. Furthermore, we demonstrate that ASTER is a powerful component in end-to-end recognition systems, for its ability to enhance the detector.

Text in natural images is of arbitrary orientations, requiring detection in terms of oriented bounding boxes. Normally, a multi-oriented text detector often involves two key tasks: 1) text presence detection, which is a classification problem disregarding text orientation; 2) oriented bounding box regression, which concerns about text orientation. Previous methods rely on shared features for both tasks, resulting in degraded performance due to the incompatibility of the two tasks. To address this issue, we propose to perform classification and regression on features of different characteristics, extracted by two network branches of different designs. Concretely, the regression branch extracts rotation-sensitive features by actively rotating the convolutional filters, while the classification branch extracts rotation-invariant features by pooling the rotation-sensitive features. The proposed method named Rotation-sensitive Regression Detector (RRD) achieves state-of-the-art performance on three oriented scene text benchmark datasets, including ICDAR 2015, MSRA-TD500, RCTW-17 and COCO-Text. Furthermore, RRD achieves a significant improvement on a ship collection dataset, demonstrating its generality on oriented object detection., accepted by CVPR 2018

Chinese is the most widely used language in the world. Algorithms that read Chinese text in natural images facilitate applications of various kinds. Despite the large potential value, datasets and competitions in the past primarily focus on English, which bares very different characteristics than Chinese. This report introduces RCTW, a new competition that focuses on Chinese text reading. The competition features a large-scale dataset with 12,263 annotated images. Two tasks, namely text localization and end-to-end recognition, are set up. The competition took place from January 20 to May 31, 2017. 23 valid submissions were received from 19 teams. This report includes dataset description, task definitions, evaluation protocols, and results summaries and analysis. Through this competition, we call for more future research on the Chinese text reading problem. The official website for the competition is http://rctw.vlrlab.net

Most state-of-the-art text detection methods are specific to horizontal Latin text and are not fast enough for real-time applications. We introduce Segment Linking (SegLink), an oriented text detection method. The main idea is to decompose text into two locally detectable elements, namely segments and links. A segment is an oriented box covering a part of a word or text line; A link connects two adjacent segments, indicating that they belong to the same word or text line. Both elements are detected densely at multiple scales by an end-to-end trained, fully-convolutional neural network. Final detections are produced by combining segments connected by links. Compared with previous methods, SegLink improves along the dimensions of accuracy, speed, and ease of training. It achieves an f-measure of 75.0% on the standard ICDAR 2015 Incidental (Challenge 4) benchmark, outperforming the previous best by a large margin. It runs at over 20 FPS on 512x512 images. Moreover, without modification, SegLink is able to detect long lines of non-Latin text, such as Chinese., Comment: To Appear in CVPR 2017

In this paper, we present a deep regression approach for face alignment. The deep regressor is a neural network that consists of a global layer and multistage local layers. The global layer estimates the initial face shape from the whole image, while the following local layers iteratively update the shape with local image observations. Combining standard derivations and numerical approximations, we make all layers able to backpropagate error differentials, so that we can apply the standard backpropagation to jointly learn the parameters from all layers. We show that the resulting deep regressor gradually and evenly approaches the true facial landmarks stage by stage, avoiding the tendency that often occurs in the cascaded regression methods and deteriorates the overall performance: yielding early stage regressors with high alignment accuracy gains but later stage regressors with low alignment accuracy gains. Experimental results on standard benchmarks demonstrate that our approach brings significant improvements over previous cascaded regression algorithms.

Aerial scene classification, which aims to automatically label an aerial image with a specific semantic category, is a fundamental problem for understanding high-resolution remote sensing imagery. In recent years, it has become an active task in the remote sensing area, and numerous algorithms have been proposed for this task, including many machine learning and data-driven approaches. However, the existing data sets for aerial scene classification, such as UC-Merced data set and WHU-RS19, contain relatively small sizes, and the results on them are already saturated. This largely limits the development of scene classification algorithms. This paper describes the Aerial Image data set (AID): a large-scale data set for aerial scene classification. The goal of AID is to advance the state of the arts in scene classification of remote sensing images. For creating AID, we collect and annotate more than 10000 aerial scene images. In addition, a comprehensive review of the existing aerial scene classification techniques as well as recent widely used deep learning methods is given. Finally, we provide a performance analysis of typical aerial scene classification and deep learning approaches on AID, which can be served as the baseline results on this benchmark.

Recognizing text in natural images is a challenging task with many unsolved problems. Different from those in documents, words in natural images often possess irregular shapes, which are caused by perspective distortion, curved character placement, etc. We propose RARE (Robust text recognizer with Automatic REctification), a recognition model that is robust to irregular text. RARE is a specially-designed deep neural network, which consists of a Spatial Transformer Network (STN) and a Sequence Recognition Network (SRN). In testing, an image is firstly rectified via a predicted Thin-Plate-Spline (TPS) transformation, into a more "readable" image for the following SRN, which recognizes text through a sequence recognition approach. We show that the model is able to recognize several types of irregular text, including perspective text and curved text. RARE is end-to-end trainable, requiring only images and associated text labels, making it convenient to train and deploy the model in practical systems. State-of-the-art or highly-competitive performance achieved on several benchmarks well demonstrates the effectiveness of the proposed model., Accepted by CVPR 2016

Many years of experimentation have led to the development and improvement of equipment and methods used to make gas source rock correlations. By crushing samples using a ball mill and directly collecting adsorbed gases in the absence of aqueous media under high vacuum conditions, most possible interference factors, such as atmospheric pollution, crushing-induced pyrolysis, and gas collection by aqueous media are eliminated. This greatly enhances the volume percentage of hydrocarbon gas in the acquired adsorbed gases, with maxima up to more than 80%. The actual measurement of carbon isotopic series can be carried out to such an extent as to be δ13C1-δ13C5. A preliminary study using newly established equipment and methods has indicated the following. (1) The carbon isotopic composition of ethane in adsorbed gases on hydrocarbon source rocks can be used to distinguish the types of source rocks. This is consistent with results obtained by using organic geochemical parameters for source rocks, and illustrates that it is highly feasible to use the carbon isotopic composition of ethane in natural gases as a parameter for distinguishing the types of source rocks. (2) The thermal evolution degree (Ro) of hydrocarbon-source rocks calculated in terms of the carbon isotopic composition of methane in adsorbed gases on hydrocarbon source rocks agrees well with the vitrinite reflectance actually measured in the source rocks. This confirms the reliability of the relationship between the carbon isotopic composition of methane in natural gases and the thermal evolution degree determined using statistics. (3) Finally, a direct gas source correlation method for natural gases has been established, and the expression of log Ro vs. δ13C1 established in terms of actually measured δ13C1 values of methane in absorbed gases. Ro values of hydrocarbon source rocks have been established as well, thus creating favorable conditions for precise oil-and-gas source correlations in exploration areas.

Objective: The objective of this study was to screen and identify differentially expressed genes in invasive bladder transitional cell carcinoma (BTCC). Methods: Voided urine samples were collected from consecutive patients with BTCC and patients under surveillance for bladder cancer recurrence; voided urine samples from patients with nonmalignant diseases served as control. We identified the differentially expressed genes by comparing urine samples of bladder carcinoma to that of the control group with suppressive subtractive hybridization (SSH) and cDNA microarray. The differentially expressed genes were verified by quantitative real-time polymerase chain reaction (QPCR). Results: From the 762 white colonies, a total of 449 positive clones were obtained in which 112 were found to be upregulated in BTCC. Sequencing and homology analysis were performed for these 112 clonies. The detection rates of some known genes (including IGF- 1, human telomerase reverse transcriptase [hTERT], bladder cancer specific nuclear matrix protein 4 [BLCA-4] and homeobox A13 [HOXA13]) for BTCC at the Ta, T1 and >T1 stages were 48%, 90% and 100%, respectively, with a specificity of 85%. The test specificity was 80% for the 30 control patients with urinary tract infections. The combination of BLCA-4 and HOXA13 could distinguish between low- and high-grade tumours, with specificity and sensitivity of 80%. Conclusion: We successfully constructed a reliable SSH library of BTCC and found that combination detection insulin-like growth factor 1 (IGF-1), hTERT, BLCA-4 and HOXA13 genes could help to evaluate BTCC at different stages.

With the rapid increase of transnational communication and cooperation, people frequently encounter multilingual scenarios in various situations. In this paper, we are concerned with a relatively new problem: script identification at word or line levels in natural scenes. A large-scale dataset with a great quantity of natural images and 10 types of widely used languages is constructed and released. In allusion to the challenges in script identification in real-world scenarios, a deep learning based algorithm is proposed. The experiments on the proposed dataset demonstrate that our algorithm achieves superior performance, compared with conventional image classification methods, such as the original CNN architecture and LLC., Comment: 5 pages, 7 figures, submitted to ICDAR 2015

Image-based sequence recognition has been a long-standing research topic in computer vision. In this paper, we investigate the problem of scene text recognition, which is among the most important and challenging tasks in image-based sequence recognition. A novel neural network architecture, which integrates feature extraction, sequence modeling and transcription into a unified framework, is proposed. Compared with previous systems for scene text recognition, the proposed architecture possesses four distinctive properties: (1) It is end-to-end trainable, in contrast to most of the existing algorithms whose components are separately trained and tuned. (2) It naturally handles sequences in arbitrary lengths, involving no character segmentation or horizontal scale normalization. (3) It is not confined to any predefined lexicon and achieves remarkable performances in both lexicon-free and lexicon-based scene text recognition tasks. (4) It generates an effective yet much smaller model, which is more practical for real-world application scenarios. The experiments on standard benchmarks, including the IIIT-5K, Street View Text and ICDAR datasets, demonstrate the superiority of the proposed algorithm over the prior arts. Moreover, the proposed algorithm performs well in the task of image-based music score recognition, which evidently verifies the generality of it., Comment: 5 figures

Objective: To study the expression of dendritic cells in human renal cell carcinoma and explore the cause, so to reveal the mechanism of escaping immune surveillance in RCC. Methods: The expressions of CD83+DCs, CD1a+DCs,VEGF and TGF-β1 in tumoral, peritumoral and normal kidney tissues of RCC in 30 cases were detected by immunohistochemistry using streptavidin/peroxidese(SP) Results: CD83+DCs were mainly located in the peritumoral areas; whereas CD1a+DCs were mainly retained within the cancer nests. The number of CD83+DCs was inversely correlated with the clinical stage (P 0.05). The expressions of CD83+DCs and CD1a+DCs have significant difference between the tumoral, peritumoral and normal kidney tissues(P 0.05). Conclusion: DC has the tendency to gathering in tumor, but because of the immunosuppressive cytokins, for example VEGF and TGF-β1, inhibits the maturation of DC, there are less mature TIDCs(CD83+TIDCs) in the tumoral tissues, they are mainly located in the peritumoral areas. This may contribute to the mechanism of escaping immune surveillance in RCC.

Taking Chang 7 shale of the Yanchang Formation in the southeastern area of the Yishan slope of the Ordos Basin as our research subject, desorption experiments were performed on more than 30 shale samples that helped this study focus on the variation of carbon isotopic composition of alkanes during the desorption process of shale gas, the possible causes of this phenomenon and its geological significance will also be discussed as follows. It was found that carbon isotopic composition became higher by 9.2‰ (from −50.1‰ to −40.9‰) for methane desorbed gas (δ 13 C 1 ), and it also became higher by 2.8‰ (from −35.5‰ to −32.7‰) for ethane (δ 13 C 2 ), but there's barely any differences for propane during the desorption process. At room temperature or constant temperature, carbon isotopic composition for both methane and ethane (δ 13 C 1 & δ 13 C 2 ) increased continuously. The values of δ 13 C 1 and δ 13 C 2 were low at first and then it gradually became high whilst the desorption process of shale gas that was optimized with the aid of the increasing temperature. The reason for this phenomenon is most likely due to the main adsorption/desorption of shale gas and diffusion migration fractionation of isotope. Additionally, these variation characteristics of methane and ethane carbon isotope composition of shale gas may be a vital reason for the relatively low carbon isotopic composition of methane compared to δ 13 C calculated by means of the vitrinite reflectance ( R O ), and this may be applied to evaluate the remaining amount of shale gas resources.