Search

Multi-person pose estimation is a fundamental yet challenging research topic for many computer vision applications. In this paper, to relieve the problem of variable pose structure and occlusion in complex scenes, we propose a novel Pose Knowledge Transfer approach for multi-person pose estimation, which doesn’t take into account deeper and wider network structure design. This approach contains a Keypoint Region Erasing (KRE) scheme and a Bi-directional Pose Knowledge Transfer (BPKT) model learning strategy for this task. The KRE encourages human pose estimator to explicitly focus learning on keypoints connectivity to robustly localize the occluded keypoint via its adjacent visible body patches. Specifically, without additional model parameters involvement, the BPKT effectively transfers the local connectivity knowledge and the global body configuration knowledge between two network with the same structure to encounter variable pose structure. Extensive experiments demonstrate that the BPKT and the KRE significantly improve the performance of a range of state-of-the-art human pose estimation models, consistently validating the effectiveness and generalization property of our model-agnostic approach on the MPII human pose dataset and the COCO keypoint benchmark.

In China, although burn treatment develops rapidly, and ranks in the forefront of the world, there is a relative shortage of burn specialists, which limits the development of burn education. In traditional curriculum of surgery education, burn surgery education accounts for few proportions, which results in the indifference to the burn surgery among medical students. To date, few research reported the application of Clinical pathway- Problem based Learning (CP-PBL) in burn surgery education. The objective of the study is to explore the teaching effect of this novel teaching method in burn surgery education.A pilot study was performed. One hundred and six students were randomly divided into a Lecture based Learning (LBL) only group (control group) and a LBL combined Clinical pathway- Problem based Learning group (observation group). A set of test was designed as evaluation criteria based on questions of burn surgery in National Medical Licensing Examination (NMLE) from 2011 to 2018.The students with Clinical pathway- Problem based Learning had better academic performances in profession theory. Type A2 and Type A3/A4 scores in the observation group were higher than those in the control group (p 0.05). The scores of the observation group were higher than those of control group in the domains of understanding and application (p 0.05). They also have higher favorable impressions of learning experience.More active approaches yield more learning and are viewed more favorable, which provides a vital message for the evolution of curriculum in Chinese medical schools.

Event recognition in surveillance video has gained extensive attention from the computer vision community. This process still faces enormous challenges due to the tiny inter-class variations that are caused by various facets, such as severe occlusion, cluttered backgrounds, and so forth. To address these issues, we propose a spatio-temporal deep residual network with hierarchical attentions (STDRN-HA) for video event recognition. In the first attention layer, the ResNet fully connected feature guides the Faster R-CNN feature to generate object-based attention (O-attention) for target objects. In the second attention layer, the O-attention further guides the ResNet convolutional feature to yield the holistic attention (H-attention) in order to perceive more details of the occluded objects and the global background. In the third attention layer, the attention maps use the deep features to obtain the attention-enhanced features. Then, the attention-enhanced features are input into a deep residual recurrent network, which is used to mine more event clues from videos. Furthermore, an optimized loss function named softmax-RC is designed, which embeds the residual block regularization and center loss to solve the vanishing gradient in a deep network and enlarge the distance between inter-classes. We also build a temporal branch to exploit the long- and short-term motion information. The final results are obtained by fusing the outputs of the spatial and temporal streams. Experiments on the four realistic video datasets, CCV, VIRAT 1.0, VIRAT 2.0, and HMDB51, demonstrate that the proposed method has good performance and achieves state-of-the-art results.

We present an algorithm to generate digital painting lighting effects from a single image. Our algorithm is based on a key observation: Artists use many overlapping strokes to paint lighting effects, i.e., pixels with dense stroke history tend to gather more illumination strokes. Based on this observation, we design an algorithm to both estimate the density of strokes in a digital painting using color geometry and then generate novel lighting effects by mimicking artists’ coarse-to-fine workflow. Coarse lighting effects are first generated using a wave transform and then retouched according to the stroke density of the original illustrations into usable lighting effects. Our algorithm is content-aware, with generated lighting effects naturally adapting to image structures, and can be used as an interactive tool to simplify current labor-intensive workflows for generating lighting effects for digital and matte paintings. In addition, our algorithm can also produce usable lighting effects for photographs or three-dimensional rendered images. We evaluate our approach with both an in-depth qualitative and a quantitative analysis that includes a perceptual user study. Results show that our proposed approach is not only able to produce favorable lighting effects with respect to existing approaches, but also that it is able to significantly reduce the needed interaction time.

A cancer vaccine has been widely applied in clinical tumor therapy as one of the main strategies of immunotherapy. However, the traditional cancer vaccine for a single antigen has a low benefit rate due to the individual differences in patients. Here, we report a R837-loaded poly(lactic-co-glycolic acid) nanovaccine coated with a calcinetin (CRT)-expressed cancer cell membrane antigen for immunotherapy. The cell membrane antigen that possessed a complete antigen array was obtained by inducing immunogenic cell death in vitro, avoiding the severe systemic toxicity of chemotherapy in vivo. The nanovaccine codelivers the adjuvant R837 and the Luc-4T1 membrane antigen, triggering a personalized immune response to the corresponding tumor. Moreover, the calcinetin exposed on the surface of the nanovaccine induces the active uptake of dendritic cells, consequently enhancing the antitumor effect. Meanwhile, the nanovaccine activates immune memory cells to provide long-term protection. Our work provides a new strategy for a clinical personalized antitumor vaccine.

Many older adults with Alzheimer's disease and patients with conditions such as spinal nerve injuries have symptoms of urinary incontinence. In order to improve their quality of life, B-mode ultrasound and drainage tubes are often used to measure the urine accumulation in the human bladder. In this paper, an ultrasonic urine system based on A-mode ultrasound is developed for measuring the urine accumulation in the human bladder. The ultrasound urine system contains three parts: the collection device, the mobile terminal, and the upper data processing software. In the collection device, the STM32 microcontroller sends out a pulse signal with a rate of 2.5MHz to excite the ultrasonic transducer through the ultrasonic transmitter circuit. Ultrasonic transducer receives the echo signals by the ultrasonic receiving circuit processing into digital signals, STM32 microcontroller transmits them to the mobile terminal. The mobile terminal transmits the digital signals to the upper data processing software for processing and receives the calculation results for display. We conducted tests on a bladder simulator. We conducted three sets of experiments and measured the transducer to baffle distance of 30mm, 40mm, and 50mm. The accuracy of the system measurements in three sets of experiments were 99.8%, 99.65%, and 99.52%. Experimental results verify the effectiveness of the system.

For a given image, the corresponding scene graph is a kind of structural expression which benefits to high-level tasks. To generate a meaningful and useful one, the existing models pay more attention on reducing the bias from long-tail distribution of dataset. However, they overlook the unimodal bias and evaluation bias from models themselves. In this paper, we construct an unbiased solution called Balanced Label and Vision for Multilabel Classification (BLVMC). BLVMC consists of two modules, label-vision grounding module (LVGM) and no graph constraint (NGC). Specially, the LVGM aims to be in equilibrium for label and vision by introducing visual information into label branch. This module reduces unimodal bias from previous models and makes them more stable. The NGC views the Scene Graph Generation (SGG) as a multilabel classification task instead of multiclass classification. Besides, the NGC uses the corresponding NGC mR@K to evaluate models. This module allows each subject-object pair to retain multi-predicates, which relieves evaluation bias. The quantitative and qualitative experiments on Visual Genome (VG) dataset demonstrate the proposed BLVMC effectively eliminates the above two biases and outperforms previous state-of-the-art models.

Scene text detection in the natural environment is widely used in real-world applications, ranging from autonomous driving, image search and assistance for the blind. However, a vast of the existing methods have limited ability to detect text instances in challenging scenes such as texts with low contrast or blur. To address the problem, we propose a novel Bi-Direction Feature Pyramid Network (BDFPN), which draws inspiration from the two-way visual information processing mechanism of human beings. Specifically, the bottom-up path is data-driven for fine details and the top-down path is task-driven for obtaining semantic information. In the top-down path, the Feature Alignment Module (FAM) is proposed to narrow the semantic differences that exist in features of adjacent levels. To combine features from two paths, we propose a novel fusion strategy named Attention Fusion Module (AFM). We conduct extensive experiments on ICDAR2015, Totaltext and MSRA-TD500 to demonstrate the effectiveness and robustness of BDFPN.

Arbitrary style transfer refers to generate a new image based on any set of existing images. Meanwhile, the generated image retains the content structure of one and the style pattern of another. In terms of content retention and style transfer, the recent arbitrary style transfer algorithms normally perform well in one, but it is difficult to find a trade-off between the two. In this paper, we propose the Deep Content Guidance Network (DCGN) which is stacked by content guidance (CG) layers. And each CG layer involves one position self-attention (pSA) module, one channel self-attention (cSA) module and one content guidance attention (cGA) module. Specially, the pSA module extracts more effective content information on the spatial layout of content images and the cSA module makes the style representation of style images in the channel dimension richer. And in the non-local view, the cGA module utilizes content information to guide the distribution of style features, which obtains a more detailed style expression. Moreover, we introduce a new permutation loss to generalize feature expression, so as to obtain abundant feature expressions while maintaining content structure. Qualitative and quantitative experiments verify that our approach can transform into better stylized images than the state-of-the-art methods.

Objective Long non-coding RNA maternally expressed gene 3 (lnc-MEG3) directly targets microRNA-21 (miR-21) to regulate the vascular microenvironment, and is closely implicated in the pathology of acute ischemic stroke (AIS). However, no research regarding the interaction of lnc-MEG3 and miR-21 in AIS patients has been conducted, to the best of our knowledge. Therefore, we performed this study to evaluate the correlation of lnc-MEG3 with miR-21, and to explore their clinical role for AIS management. Methods A total of 170 AIS patients and 100 controls with at least two high-risk factors for stroke were enrolled. The expression of lnc-MEG3 and miR-21 in peripheral blood mononuclear cells was detected by reverse transcription-quantitative polymerase chain reaction. Results Lnc-MEG3 expression was increased in AIS patients and could differentiate AIS patients from controls using receiver operating characteristic (ROC) curve analysis with area under the curve (AUC) of 0.874 and a 95% confidence interval (CI) of 0.833–0.914. Lnc-MEG3 expression was positively correlated with tumor necrosis factor (TNF)-α, interleukin (IL)-6, IL-17A and the National Institutes of Health Stroke Scale (NIHSS) score, and its high expression was also correlated with elevated accumulating recurrence rate in AIS patients. In addition, lnc-MEG3 expression was negatively correlated with miR-21 expression in AIS patients. Regarding miR-21, it was reduced in AIS patients and could differentiate AIS patients from controls with AUC of 0.889 (95% CI: 0.850–0.927). Also, miR-21 expression was negatively correlated with TNF-α, IL-17A, NIHSS score and accumulating recurrence rate in AIS patients. Conclusion Lnc-MEG3 is negatively correlated with miR-21, and both factors are related to disease risk, inflammatory cytokines, disease severity and recurrence risk of AIS.

Nowadays content-based video event detection faces great challenges due to complex scenes and blurred actions in surveillance videos. To alleviate these challenges, we propose a novel spatial-temporal architecture of deep convolutional neural networks for this task. By taking advantage of spatial-temporal information, we fine-tune two-stream networks, and then, fuse spatial and temporal features at convolution layers using a 2D pooling fusion method to enforce the consistence of spatial-temporal information. Based on the two-stream networks and spatial-temporal layer, a triple-channel model is obtained. Furthermore, we implement trajectory-constrained pooling to deep features and hand-crafted features to combine their merits. A fusion method on triple-channel yields the final detection result. The experiments on two benchmark surveillance video data sets including VIRAT 1.0 and VIRAT 2.0, which involve a suit of challenging events, such as person loading an object to a vehicle or person opening a vehicle trunk, manifest that the proposed method can achieve superior performance compared with the state-of-the-art methods on these event benchmarks.

Flat filling is a critical step in digital artistic content creation with the objective of filling line arts with flat colors. We present a deep learning framework for user-guided line art flat filling that can compute the "influence areas" of the user color scribbles, i.e., the areas where the user scribbles should propagate and influence. This framework explicitly controls such scribble influence areas for artists to manipulate the colors of image details and avoid color leakage/contamination between scribbles, and simultaneously, leverages data-driven color generation to facilitate content creation. This framework is based on a Split Filling Mechanism (SFM), which first splits the user scribbles into individual groups and then independently processes the colors and influence areas of each group with a Convolutional Neural Network (CNN). Learned from more than a million illustrations, the framework can estimate the scribble influence areas in a content-aware manner, and can smartly generate visually pleasing colors to assist the daily works of artists. We show that our proposed framework is easy to use, allowing even amateurs to obtain professional-quality results on a wide variety of line arts.

We present a framework to generate manga from digital illustrations. In professional mange studios, the manga create workflow consists of three key steps: (1) Artists use line drawings to delineate the structural outlines in manga storyboards. (2) Artists apply several types of regular screentones to render the shading, occlusion, and object materials. (3) Artists selectively paste irregular screen textures onto the canvas to achieve various background layouts or special effects. Motivated by this workflow, we propose a data-driven framework to convert a digital illustration into three corresponding components: manga line drawing, regular screen-tone, and irregular screen texture. These components can be directly composed into manga images and can be further retouched for more plentiful manga creations. To this end, we create a large-scale dataset with these three components annotated by artists in a human-in-the-loop manner. We conduct both perceptual user study and qualitative evaluation of the generated manga, and observe that our generated image layers for these three components are practically usable in the daily works of manga artists. We provide 60 qualitative results and 15 additional comparisons in the supplementary material. We will make our presented manga dataset publicly available to assist related applications.

Background: At present, with the continuous development of the epidemic, scholars at home and abroad have paid great attention to Corona Virus Discrease 2019(COVID-19), especially with SARS, comprehensive biology combined and carried out a series of research and discussion. Research on such topics has gradually increased, but the hot topics of research still not clear. Purpose: The goal of the research was to perform a systematic review on the use of biomedicine in medical research with the aim of understanding the global progress on COVID-19 research outcomes, content, methods, and study groups involved. Methods: The SARS documents were retrieved in the core collection of Web of Science, and analyze the data with Excel and VOSviewer.to perform bibliometric analysis of publication trends, author orders, countries, institutions, collaboration relationships, research hot spots, diseases studied, and research methods. Data Synthesis: A total of 50,744 original research articles were included. The number of articles published in the first 3 years showed an overall upward trend and then gradually declined to basically the same. It accounts for 2.087% of the total number of posts, reaching a peak of 31050 in 2020, accounting for 61.190% of the total. From 6,976 journals, 18 journals with more than 250 articles published, more than half of the journals are in JCR Zone 1, and most of the influence factors are between 2-7 points; the included literature involves 61684 authors, and most of the authors are scattered. The included literature involved 9738 institutions. Among the first 18 institutions, more than 80% were colleges and hospitals. A total of 37,817 keywords are listed, with an average of 2.65 keywords. The top 7 in frequency of use are acute lung injury, acute respiratory distress syndrome, respiratory-distress-syndrome, aras, mechanical ventilation, and mortality. The centrality is more than 7000, and the number of occurrences is more than 1000. The research direction of SARS focuses on: RESPIRATORY SYSTEM, INFECTIOUS DISEASES, PUBLIC ENVIRONMENTAL OCCUPATIONAL HEALTH, BIOCHEMISTRY MOLECULAR BIOLOGY and HEALTH CARE SCIENCES SERVICES, which are all higher than 27% of the total, and the records are higher than 13,900. High-frequency keyword clustering and overlay view analysis: (1) In terms of mechanism research, gene structure, virus detection, outbreak prevention and protection, etc. are the major research hotspots. (2) In terms of target therapy, animal molecular experiments such as protein, activation, modle mode, potent inhibitors, laboratory tests, and immune response are the research hotspots. Conclusions: Based on the research of SARS literature, preliminary research suggests that COVID-19 research will increase rapidly by multiples, and the popularity of research continues. In terms of public health and epidemic prevention, the world still needs to form a stronger team. It needs to do a good job of inheritance, achieve full prevention and control performance, innovate results, and explore potential areas with reference to common word clustering and knowledge graphs to promote the emergence of authoritative innovation. Research trends will expand from the development of diagnostics and treatments to the human-society-biological environment, making important contributions to the global fight against the epidemic.

Semi-supervised video object segmentation aims at segmenting the target of interest throughout a video sequence when only the annotated mask of the first frame is given. A feasible method for segmentation is to capture the spatial-temporal coherence between frames. However, it may suffer from mask drift when the spatial-temporal coherence is unreliable. To relieve this problem, we propose an encoder-decoder-recurrent model for semi-supervised video object segmentation. The model adopts a U-shape architecture that combines atrous convolution and ConvLSTM to establish the coherence in both the spatial and temporal domains. Furthermore, the weight ratio for each block is also reconstructed to make the model more suitable for the VOS task. We evaluate our method on two benchmarks, DAVIS-2017 and Youtube-VOS, where state-of-the-art segmentation accuracy with a real-time inference speed of 21.3 frames per second on a Tesla P100 is obtained.

Visual dialog is a typical multi-modal task which involves both vision and language. Nowadays, it faces two major difficulties. In this paper, we propose the Integrating Historical States and Co-attention (HSCA) for visual dialog to solve them. It includes two main modules, Co-ATT and MATCH. Specifically, the main purpose of the Co-ATT module is to guide the image with questions and answers in the early stage to get more specific objects. It tackles the first difficulty of the temporal sequence issue in historical information which may influence the precise answer for multi-round questions. The MATCH module is, based on a question with pronouns, to retrieve the best matching historical information block. It overcomes the second difficulty of the visual reference problem which requires to solve pronouns referring to unknowns in the text message and then to locate the objects in the given image. We quantitatively and qualitatively evaluate our model on VisDial v1.0, at the same time, ablation studies are carried out. The experimental results demonstrate that HSCA outperforms the state-of-the-art methods in many aspects.

Multi-person pose estimation is a fundamental yet challenging research topic for many computer vision applications. It is difficult to achieve accurate localization results due to occlusion and complex background. In this paper, we propose a novel multi-person pose estimation approach with information complement and attention refinement residual module. To recover occlusion, the complementary features with multi-scale semantics information are extracted by our proposed Information Complement Module (ICM). To effectively discover the channel relationship and selectively highlight task-related regions in the feature maps, we design an Attention Refinement Residual Bottleneck (ARRB) module, which is an extension of residual unit with attention mechanism. We conduct ablation studies to investigate the efficacy of our method and compare it with the state-of-the-art methods on the COCO keypoint benchmark. Experimental results demonstrate that the selective complementary features are effective for multi-person pose estimation.

Analysis of pose-level information is critical for human action recognition. For fast visual perception, we propose to encode human pose with discriminative information in the pose feature space, which needs low computational cost. The main idea is that the common poses that exist in different actions have low discrimination while the discriminative poses exist in few actions. Therefore, we employ the Gaussian Mixture Model to cluster the feature vectors of all poses in all classes of actions, and then quantify the discrimination of each pose as well as its contribution to each action class. Meanwhile, we calculate the correlation between the clusters and action classes as a rating matrix. The query action is inferred by simply calculating the Gaussion weights of poses and multiply the rating matrix. The framework runs without deep neural networks, which is low cost for application on low power devices. The proposed method is validated on three benchmark human action datasets, and the experimental results show good performances of the proposed method. The inferring time of each query is only 24ms on a 2.5 GHz CPU that indicats the efficiency of our method.

Background In the past two decades, chronic non-communicable diseases have become the leading disease burden, and cardio-cerebrovascular diseases (CCVD) are the main causes of death in chronic diseases. It has become the focus of global public health attention, in this study, System of Health Accounts 2011 (SHA 2011) is used to calculate health expenditure, discuss its economic burden, and put forward countermeasures. Methods Data were collected by multi-stage stratified random sampling and the medical expenses of patients with CCVD were calculated based on SHA 2011, from the dimensions of the financing plan, institutional flow, and service function. Correlation and regression analysis were conducted by controlling factors influencing hospitalization expenses. All analysis were conducted by software SPSS. Results The current health expenditure (CHE) of CCVD in Dalian was 3.986 billion Yuan, accounting for 12.88% of the CHE (30.947 billion Yuan). The current curative expenditure (CCE) of CCVD was 2.947 billion Yuan. 40.39% of CCVD financing came from social medical insurance, and the proportion of family health financing was higher (39.06%). The expenditures were consumed by general hospitals and elderly patients. Conclusions The expenditure burden of CCVD in Dalian was massive, and wasted the health resource severely. It is necessary for the government to adjust the financing structure, reallocate the expenses of CCVD, and make institutional flow and functional distribution more reasonable.

This paper investigates the differential effects of corporate social responsibility (CSR) dimensions on corporate financial performance (CFP) across sectors in China. This research uses a unique data set provided by China Stock Market and Accounting Research (CSMAR), showing expenditure on CSR programs from 568 Chinese publicly traded firm-year observations from 2008 to 2017. Compared to previous studies using scores produced by extra-financial rating agencies, this research quantifies CSR efforts by corporate expenditure on CSR practices, which offers quantitative and precise information in explaining the CSR-CFP link. The results show that the dimension of the environment has negative effects on financial performance in capital-intensive manufacturing industries. The impact of HR expenditure on CFP is negative in the tertiary sector and resourceintensive manufacturing industries. However, CSR investments in the community are positively related to financial performance in resource-intensive industries and other secondary sector (mining, construction, and utilities). Firms, in general, could gain benefits when spending more on business and financial stakeholders.

Despite recent remarkable progress in image generation from layout, synthesizing vivid images with recognizable objects remains a challenging problem, object distortion and color imbalance occasionally happened in the generated images. To overcome these limitations, we propose a novel approach called Pair-wise Relativistic average Generative Adversarial Network (P-RaGAN) which includes a pair-wise relativistic average discriminator for enhancing the generative ability of network. We also introduce a consistency loss into our model to keep the consistency of original latent code and reconstructed or generated latent code for reducing the scope of solution space. A series of ablation experiments demonstrate the capability of our model in the task from layout to image on the complicated COCO-stuff and Visual Genome datasets. Extensive experimental results show that our model outperforms the state-of-the-art methods.

As a conversational intelligence task, visual dialog entails answering a series of questions grounded in an image, using the dialog history as context. To generate correct answers, the comprehension of the semantic dependencies among implicit visual and textual contents is critical. Prior works usually ignored the underlying relation and failed to infer it reasonably. In this paper, we propose a Visual-Textual Alignment for Graph Inference (VTAGI) network. Compared with other approaches, it makes up the lack of structural inference in visual dialog. The whole system consists of two modules, Visual and Textual Alignment (VTA) and Visual Graph Attended by Text (VGAT). Specially, the VTA module aims at representing an image with a set of integrated visual regions and corresponding textual concepts, reflecting certain semantics. The VGAT module views the visual features with semantic information as observed nodes and each node learns the relationship with others in visual graph. We also qualitatively and quantitatively evaluate the model on VisDial v1.0 dataset, showing our VTAGI outperforms previous state-of-the-art models.

Synthesizing realistic images from coarse layout is a challenging problem in computer vision. For generating the image with multiple objects, there still exists the problem of the shortage of object details and certain artifacts in existing methods. In this paper, we propose an image generation approach based on the Modified Variational Lower Bound and Sub-pixel Sampling (MVLBSS). Firstly, we modify variational lower bound to form the enhanced latent code with the semantic of object appearance. The change of variational lower bound is implemented by introducing the object category information in the crop estimator and layout encoder to capture the details of object appearance. Secondly, to reduce artifacts, we propose a sub-pixel sampling module with up-sampling and convolution operation. Lastly, our proposed approach is trained in an end-to-end manner to the global optimal image generation. The comparison experimental results with the state-of-the-art methods on both COCO-stuff and Visual Genome datasets show that the proposed method can effectively improve the quality of generation image. Moreover, ablation experiments demonstrate that the improved strategy is effective for enhancing image details and eliminating artifacts.

Optimization-based Image smoothing is routinely formulated as the game between a smoothing energy and an appearance preservation energy. Achieving adequate smoothing is a fundamental goal of these Image smoothing algorithms. We show that partially “erasing” the appearance preservation facilitate adequate Image smoothing. In this paper, we call this manipulation as Erasing Appearance Preservation (EAP). We conduct an user study, allowing users to indicate the “erasing” positions by drawing scribbles interactively, to verify the correctness and effectiveness of EAP. We observe the characteristics of human-indicated “erasing” positions, and then formulate a simple and effective 0-1 knapsack to automatically synthesize the “erasing” positions. We test our synthesized erasing positions in a majority of Image smoothing methods. Experimental results and large-scale perceptual human judgments show that the EAP solution tends to encourage the pattern separation or elimination capabilities of Image smoothing algorithms. We further study the performance of the EAP solution in many image decomposition problems to decompose textures, shadows, and the challenging specular reflections. We also present examinations of diversiform image manipulation applications like texture removal, retexturing, intrinsic decomposition, layer extraction, recoloring, material manipulation, etc. Due to the widespread applicability of Image smoothing, the EAP is also likely to be used in more image editing applications.

Sketch or line art colorization is a research field with significant market demand. Different from photo colorization which strongly relies on texture information, sketch colorization is more challenging as sketches may not have texture. Even worse, color, texture, and gradient have to be generated from the abstract sketch lines. In this paper, we propose a semi-automatic learning-based framework to colorize sketches with proper color, texture as well as gradient. Our framework consists of two stages. In the first drafting stage, our model guesses color regions and splashes a rich variety of colors over the sketch to obtain a color draft. In the second refinement stage, it detects the unnatural colors and artifacts, and try to fix and refine the result. Comparing to existing approaches, this two-stage design effectively divides the complex colorization task into two simpler and goal-clearer subtasks. This eases the learning and raises the quality of colorization. Our model resolves the artifacts such as water-color blurring, color distortion, and dull textures. We build an interactive software based on our model for evaluation. Users can iteratively edit and refine the colorization. We evaluate our learning model and the interactive system through an extensive user study. Statistics shows that our method outperforms the state-of-art techniques and industrial applications in several aspects including, the visual quality, the ability of user control, user experience, and other metrics.

To solve the challenging person re-identification problem, great efforts have been devoted to feature representation and metric learning. However, existing feature extractors are either stripe-based or dense-block-based, the fine details and coarse appearance are not well integrated. What is more, the metrics are generally learned independently from distance view or bilinear similarity view. Few works have exploited the mutual complementary effects of their combination. To address these issues, we propose a new feature representation termed enhanced Local Maximal Occurrence (eLOMO) which fuses a new overlapping-stripe-based descriptor with the Local Maximal Occurrence (LOMO) extracted from dense blocks. Such integration makes eLOMO resemble the coarse-to-fine recognition mechanism of human vision system, thus it can provide a more discriminative descriptor for re-identification. Besides, we show the advantages of learning generalized similarity by combining the Mahalanobis distance and bilinear similarity together. Specifically, we derive a logistic metric learning method to jointly learn a distance metric and a bilinear similarity metric, which exploits both the distance and angle information from training data. Taking advantage of learning in the intra-class subspace, the proposed method can be solved efficiently by coordinate descent optimization. Experiments on four challenging datasets including VIPeR, PRID450S, QMUL GRID, and CUHK01, show that the proposed method outperforms the state-of-the-art approaches significantly.

As a cross-media intelligence task, visual dialog calls for answering a sequence of questions based on an image, using the dialog history as context. To acquire correct answers, the exploration of the semantic dependencies among potential visual and textual contents becomes vital. Prior works usually ignored the underlying knowledge hidden in internal and external textual-visual relationships, which resulted in unreasonable inferring. In this paper, we propose an Aligning Vision-Language for Graph Inference (AVLGI) in visual dialog by combining the internal context-aware information and the external scene graph knowledge. Compared with other approaches, it makes up the lack of structural inference in visual dialog. So the whole system consists of three modules, Inter-Modalities Alignment (IMA), Visual Graph Attended by Text (VGAT) and Combining Scene Graph and Textual Contents(CSGTC). Specifically, the IMA module aims at representing an image with a set of integrated visual regions and corresponding textual concepts, reflecting certain semantics. And the VGAT module views the visual features with semantic information as observed nodes and measures the weight of importance between each two nodes in visual graph. The CSGTC supplements various relationships between visual objects by introducing additional information of the scene graph. We also qualitatively and quantitatively evaluate the model on VisDial v1.0 dataset, showing our AVLGI outperforms previous state-of-the-art models.

Video action recognition is an important topic in computer vision tasks. Most of the existing methods use CNN-based models, and multiple modalities of image features are captured from the videos, such as static frames, dynamic images, and optical flow features. However, these mainstream features contain much static information including object and background information, where the motion information of the action itself is not distinguished and strengthened. In this work, a new kind of motion feature is proposed without static information for video action recognition. We propose a quantization of motion network based on the bag-of-feature method to learn significant and discriminative motion features. In the learned feature map, the object and background information is filtered out, even if the background is moving in the video. Therefore, the motion feature is complementary to the static image feature and the static information in the dynamic image and optical flow. A multi-stream classifier is built with the proposed motion feature and other features, and the performance of action recognition is enhanced comparing to other state-of-the-art methods.

// Chunping Liu 1, * , Qiuyang Zhao 1, * , Zhi Li 1 , Shuntao Wang 1 , Yiquan Xiong 1 , Zeming Liu 1,* and Tao Huang 1 1 Department of Breast and Thyroid Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430022, People’s Republic of China * These authors have contributed equally to this work Correspondence to: Tao Huang, email: huangtaowhunion@163.com Zeming Liu, email: 6myt@163.com Keywords: mixed subtype thyroid carcinoma; prognosis; SEER; PSM Received: August 06, 2017 Accepted: August 29, 2017 Published: September 23, 2017 ABSTRACT The prognosis of patients with mixed subtype thyroid cancer (MSTC) is unclear. The present study compared the prognoses of MSTC, papillary thyroid cancer (PTC), and follicular thyroid cancer (FTC) to provide a new perspective regarding the treatment guidelines for these diseases. We evaluated data from patients with thyroid cancer who were included in the Surveillance, Epidemiology, and End Results database between 2004 and 2013. Patient mortality was evaluated using Cox proportional hazards regression analyses and Kaplan-Meier analyses with log-rank tests. The univariate Cox regression analysis showed that the cancer-specific survival rate for MSTC was lower than that for PTC and FTC. However, in the multivariate Cox regression analysis, the cancer-specific survival rate for MSTC was similar to that for PTC and FTC. Before matching influence factors, the cancer-specific survival rate for MSTC was lower than that for PTC and FTC. However, after propensity score matching for relevant factors, the cancer-specific survival rate for MSTC was also similar to that for PTC and FTC. Our result would be beneficial and provide a guideline for the understanding of MSTC.

The prognostic significance of gender remains controversial for papillary thyroid carcinoma (PTC). In this study, we investigated the associations between gender and prognosis in a large cohort of patients with PTC or PTMC that was diagnosed in 2010–2013 and recorded in the Surveillance, Epidemiology, and End Results cancer registry. The mean ± standard deviation duration of survival for all patients with PTC during the study period was 21.47 ± 14.04 months. In Kaplan-Meier analyses of the entire cohort of PTC patients, survival curves for all-cause death and cancer-specific death declined more sharply for men than for women. Similar results were observed in analyses of patients with PTCs > 1 cm and PTMC. After adjusting for potential confounders, hazard rates indicated significantly elevated all-cause mortality for men in analyses of all PTCs, PTCs > 1 cm, and PTMCs. However, in a confounder-adjusted analysis of patients with PTMC, the hazard rate did not indicate significantly higher mortality for men than for women. Our study demonstrated that male gender is an independent poor prognostic factor for all PTCs and for PTCs > 1 cm. However, gender is not an independent prognostic factor for cause-specific survival in PTMC.

// Chunping Liu 1, * , Qiuyang Zhao 1, * , Wen Zeng 2 , Chen Chen 1 , Jie Ming 1 , Shuntao Wang 1 , Yiquan Xiong 1 , Chao Zhang 3 , Tianwen Chen 4 , Zeming Liu 1 and Tao Huang 1 1 Department of Breast and Thyroid Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430022, China 2 Department of Ophthalmology, Zhongnan Hospital, Wuhan University, Wuhan, China 3 Department of Cardiovascular Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China 4 Department of Breast and Thyroid Surgery, Affiliated Nanshan Hospital, Guangdong Medical University, Shenzhen, China * These authors have contributed equally to this work Correspondence to: Tao Huang, email: huangtaowhunion@163.com Zeming Liu, email: 6myt@163.com Keywords: oxyphilic cell papillary thyroid carcinoma, prognosis, SEER, PSM Received: June 11, 2017 Accepted: July 26, 2017 Published: August 18, 2017 ABSTRACT The prognosis of oxyphilic cell papillary thyroid carcinoma (OCPTC) remains unclear. The aim of this study was to investigate the prognosis of OCPTC and provide a new perspective on treatment guidelines for these patients. We investigated a large cohort of DTC patients from the Surveillance, Epidemiology, and End Results (SEER) database between 2004 and 2013. Patient mortality was examined by Kaplan-Meier analyses with log-rank tests and Cox proportional hazards regression analyses. In the study cohort, the rate of cancer-specific mortality per 1000 person-years for OCPTC was lower than that for classic papillary thyroid cancer (CPTC) and follicular thyroid cancer (FTC). According to the multivariate Cox regression model, the cancer-specific and all-cause mortality rates of OCPTC were similar to that of CPTC and FTC. The cancer-specific survival rate in patients with OCPTC was higher than that in patients with FTC, but similar to patients with CPTC, after matching for influential factors using propensity score matching analysis. The unanticipated prognosis provided new implications for the treatment of patients with OCPTC.

Tumor-associated inflammation plays a critical role in facilitating tumor growth, invasion and metastasis. Our previous study showed Aflatoxin G1 (AFG1) could induce lung adenocarcinoma in mice. Chronic lung inflammation associated with superoxide dismutase (SOD)-2 upregulation was found in the lung carcinogenesis. However, it is unclear whether tumor-associated inflammation mediates SOD-2 to contribute to cell invasion in AFG1-induced lung adenocarcinoma. Here, we found increased SOD-2 expression associated with vimentin, α-SMA, Twist1, and MMP upregulation in AFG1-induced lung adenocarcinoma. Tumor-associated inflammatory microenvironment was also elicited, which may be related to SOD-2 upregulation and EMT in cancer cells. To mimic an AFG1-induced tumor-associated inflammatory microenvironment in vitro, we treated A549 cells and human macrophage THP-1 (MΦ-THP-1) cells with AFG1, TNF-α and/or IL-6 respectively. We found AFG1 did not promote SOD-2 expression and EMT in cancer cells, but enhanced TNF-α and SOD-2 expression in MΦ-THP-1 cells. Furthermore, TNF-α could upregulate SOD-2 expression in A549 cells through NF-κB pathway. Blocking of SOD-2 by siRNA partly inhibited TNF-α-mediated E-cadherin and vimentin alteration, and reversed EMT and cell migration in A549 cells. Thus, we suggest that tumor-associated inflammation mediates SOD-2 upregulation through NF-κB pathway, which may contribute to EMT and cell migration in AFG1-induced lung adenocarcinoma.Introduction.

// Chunping Liu 1, * , Jie Ming 1, * , Wen Zeng 2 , Shuntao Wang 1 , Yiquan Xiong 1 , Qiuyang Zhao 1 , Xingjie Yin 1 , Zeming Liu 1 and Tao Huang 1 1 Department of Breast and Thyroid Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China 2 Department of Ophthalmology, Zhongnan Hospital, Wuhan University, Wuhan, China * These authors have contributed equally to this work Correspondence to: Tao Huang, email: huangtaowh@163.com Zeming Liu, email: 6myt@163.com Keywords: differential thyroid cancer, T0 stage, prognosis, SEER Received: May 23, 2017 Accepted: July 13, 2017 Published: August 07, 2017 ABSTRACT The prognosis of T0 stage differentiated thyroid cancer (DTC) remains unclear. This study aimed to investigate the prognosis of T0 stage DTC patients to provide a new perspective on treatment guidelines for these patients. We investigated a large cohort of DTC patients from the Surveillance, Epidemiology, and End Results (SEER) database between 2004 and 2013. Patient survival curves were examined by Kaplan-Meier analyses with log-rank tests and Cox proportional hazards regression analyses. In the study cohort, the rate of cancer-specific mortality per 1000 person-years for T0 was higher than T1–T3, but lower than T4. The all-cause mortality for T0 patients was higher than all other stages (T1–T4). Multivariate Cox regression modeling showed that T0 had a significant risk for cancer-specific mortality when compared to T1 and T4, but not T2 or T3, after adjustment for other risk factors. For all-cause mortality, T0 showed a significant risk for all-cause mortality when compared to T4, but not T1–T3 stage patients. Similar results were obtained after matching for influential factors using propensity scored matching analysis. The unanticipated prognosis of T0 stage DTC patients was found to be not better than of other stage DTC patients, providing new implications for the treatment of T0 stage DTC patients.

Distance metric learning has achieved great success in person re-identification. Most existing methods that learn metrics from pairwise constraints suffer the problem of imbalanced data. In this study, the authors present a large margin relative distance learning (LMRDL) method which learns the metric from triplet constraints, so that the problem of imbalanced sample pairs can be bypassed. Different from existing triplet-based methods, LMRDL employs an improved triplet loss that enforces penalisation on the triplets with minimal inter-class distance, and this leads to a more stringent constraint to guide the learning. To suppress the large variations of pedestrian's appearance in different camera views, the authors propose to learn the metric over the intra-class subspace. The proposed method is formulated as a logistic metric learning problem with positive semi-definite constraint, and the authors derive an efficient optimisation scheme to solve it based on the accelerated proximal gradient approach. Experimental results show that the proposed method achieves state-of-the-art performance on three challenging datasets (VIPeR, PRID450S, and GRID).

The current American Joint Committee (AJCC) on Cancer TNM classification does not describe the treatment of multifocal papillary thyroid microcarcinomas (PTMCs) with a total tumour diameter (TTD) >1 cm. Herein, we investigated this PTMC subgroup in terms of extrathyroidal extension (ETE), local infiltration, central lymph node metastasis (LNM), and prognosis. Consecutive patients (n = 1102) were identified and the proportions of LNM, ETE, and local infiltration were similar between PTCs with a unifocal tumour diameter >1 cm and ≤2 cm and PTMCs with a multifocal TTD >1 cm and ≤2 cm. The proportions of LNM, ETE, and local infiltration were also similar between PTMCs with a unifocal diameter ≤1 cm vs. multifocal TTD ≤1 cm. However, when comparing PTMCs with a unifocal diameter ≤1 cm vs. multifocal TTD >1 cm, significant differences were observed. In the Kaplan-Meier analysis, significant differences were observed between PTMCs with a unifocal diameter ≤1 cm vs. multifocal TTD >1 cm and multifocal TTD ≤1 cm vs. multifocal TTD >1 cm. Accordingly, TTD may represent a more accurate criterion for tumour size of PTCs and should be considered in the revised AJCC staging system.

Three foreseeable trends are becoming so important in the coming world of intelligence, continuing emergences of new knowledge, more obscure boundaries of disciplines, and high integrations of sciences and technologies (sci-tech). How to make a sci-tech breakthrough by promoting intersection and convergence of disciplines has become a vital national strategy. However, interdisciplinary research in China is still at an early stage and discipline construction including artificial intelligence has developed slowly, which results in a shortage of intelligent technologies and human resources. It is glad to know that some Chinese sci-tech societies have initiatively attempted to promote and to some extent achieved discipline integration and intelligence innovation through building alliances characterized as openness, hub, and platform, connecting universities, institutions, enterprises and other agencies. By using the method of Qualitative Data Analysis, this paper studied the cases of 6 alliances comprised by over 70 sci-tech societies in China, such as Alliance of Intelligent Manufacturing Societies and Alliance of Information Technology Societies and verified the findings by the interviews of several managers of the alliances. The result showed the models and paths of organizational innovation of the societies in terms of cross discipline exploration, technology management optimization, and talent growing-up promotion.

As a research hotspot of multimodal media analysis, referring expression comprehension locates the referred object region in an image by mapping a natural language. Though the localizing accuracy of similar objects is often distorted by the presence or absence of supporting objects in the referring expression, we propose a referring expression comprehension method via co-attention and visual context. For lacking supporting objects in referring expression, we propose co-attention to enhance the attention on attributes for the subject module. For existing supporting objects, we introduce visual context to explore the latent link between the candidate object and its supporters. Experiments on three datasets RefCOCO, RefCOCO+, and RefCOCOg, show that our approach outperforms published approaches by a considerable margin.

Better capturing the interactions of different modality is a hot research topic in visual question answering (VQA) recently. Inspired by human vision information processing, a method of VQA based on intra-modality features interactive with self-attention mechanism (IMFI-SA) is proposed. We adopted object-level features with bottom-up attention instead of feature mapping to extract the fine-grained information in images. Moreover, the interactions of intra-modality in the question and the image modality is also extracted by proposed IMFI-SA model respectively. Finally, we combined the enhanced object-level features interaction using top-down cross-attention and the question features interaction to predict the answer given a question and image. Experimental results on the VQA2.0 dataset show that the proposed method is superior to the existing method in the reasoning answer generating, especially in counting problems.

// Zeming Liu 1, * , Wen Zeng 2, * , Tianwen Chen 1 , Yawen Guo 1 , Chao Zhang 3 , Chunping Liu 1, * , Tao Huang 1, * 1 Department of Breast and Thyroid Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China 2 Department of Ophthalmology, Zhongnan Hospital, Wuhan University, Wuhan, Hubei, China 3 Department of Cardiovascular Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China * These authors contributed equally to this work Correspondence to: Tao Huang, email: huangtaowh@163.com Chunping Liu, email: fac6myt@163.com Keywords: papillary thyroid carcinoma, tall cell variant, clinicopathological features, mortality risk, meta–analysis Received: November 01, 2016 Accepted: December 12, 2016 Published: December 21, 2016 ABSTRACT Papillary thyroid cancer (PTC) accounts for 80–90% of all thyroid malignancies. The tall cell variant (TCV) is a rare aggressive histotype of PTC. We performed a meta-analysis to compare the clinicopathological characteristics and prognostic factors of TCV with those of classical papillary thyroid carcinoma (cPTC). A literature search was performed using the PubMed and EMBASE databases using Medical Subject Headings and keywords. Twenty studies that included 1871 patients with TCV and 75323 patients with cPTC were included in our meta-analysis. Odds ratios and confidence intervals were calculated for each study. Patients with TCV were associated with multifocality, higher TNM stage, extrathyroidal extension, vascular invasion, lymph node metastasis, distant metastasis, BRAF mutation, disease-specific survival, and overall survival. We found that TCV cases were associated with more aggressive clinicopathological characteristics and poorer prognoses than cPTC cases were. Our results suggest that TCV is a high-risk PTC that warrants aggressive treatment and follow-up strategies.

Yusufu Maimaiti,1,2,* Zeming Liu,1,* Jie Tan,1 Kelimu Abudureyimu,2 Bangxing Huang,3 Chunping Liu,1 Yawen Guo,1 Changwen Wang,1 Xiu Nie,3 Jing Zhou,1 Tao Huang1 1Department of Breast and Thyroid Surgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 2Department of General Surgery, Research Institute of Minimally Invasive, People’s Hospital of Xinjiang Uygur Autonomous Region, Urumqi, 3Department of Pathology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, People’s Republic of China *These authors contributed equally to this work Background: Proteins in the cofilin pathway regulate actin dynamics and may be involved in cancer cell migration and invasion. However, there are no direct data that suggest that dephosphorylated cofilin can affect breast cancer prognosis.Methods: We assessed the expressions of cofilin and phosphorylated cofilin (P-cofilin) in breast cancer tissue microarrays (290 patients, mean follow-up: 95.7±2.49months) to evaluate dephosphorylated cofilin and its relationship with breast cancer prognosis. The associations of pathological characteristics with cumulative survival were evaluated using Kaplan–Meier analysis.Results: Univariate analyses revealed that overall survival was associated with cofilin levels, Ncategory, TNM stage, estrogen receptor status, progesterone receptor status, and molecular subtypes. Cofilin status and TNM stage independently affected overall survival, although P-cofilin expression was not associated with patient survival. In the P-cofilin-negative subgroup, cofilin expression was significantly associated with patient survival, although cofilin expression was not significantly associated with patient survival in the P-cofilin-positive subgroup. We further analyzed the P-cofilin-negative cases and found that Ki-67 expression was significantly elevated in the subgroup that was strongly positive for cofilin (P=0.002).Conclusion: Among P-cofilin-negative patients with breast cancer, cofilin expression defines a population of patients with lower overall survival, which suggests that dephosphorylated cofilin expression might predict the prognosis in cases of P-cofilin-negative breast cancer. Furthermore, our results suggest that inhibitors of dephosphorylated cofilin expression may provide therapeutic benefits in patients with breast cancer. Keywords: cofilin, P-cofilin, poor prognosis, breast cancer

Feature fusion is widely used in various neural network-based visual recognition tasks, such as object detection, to enhance the quality of feature representation. It is common practice for both the one-stage object detectors and the two-stage object detectors to implement feature fusion in feature pyramid networks (FPN) to enhance the capacity to detect objects of different scales. In this work, we propose a novel and efficient feature fusion unit, which is referred to as the Split and Combine (SC) Block, that splits the input feature maps into several parts, then processes these sub-feature maps with different emphasis, and finally gradually concatenates the outputs one-by-one. The SC block implicitly encourages the network to focus on features that are more important to the task, thus improving network efficiency and reducing inference computations. In order to prove our analysis and conclusions, a backbone network and an FPN employing this technique are assembled into a one-stage detector and evaluated on the MS COCO dataset. With the newly introduced SC block and other novel training tricks, our detector achieves a good speed-accuracy trade-off on COCO test-dev set, with 37.1% AP (average precision) at 51 FPS and 38.9% AP at 40 FPS.

Hand gesture recognition is a popular topic in computer vision and makes human-computer interaction more flexible and convenient. The representation of hand gestures is critical for recognition. In this paper, we propose a new method to measure the similarity between hand gestures and exploit it for hand gesture recognition. The depth maps of hand gestures captured via the Kinect sensors are used in our method, where the 3D hand shapes can be segmented from the cluttered backgrounds. To extract the pattern of salient 3D shape features, we propose a new descriptor-3D Shape Context, for 3D hand gesture representation. The 3D Shape Context information of each 3D point is obtained in multiple scales because both local shape context and global shape distribution are necessary for recognition. The description of all the 3D points constructs the hand gesture representation, and hand gesture recognition is explored via dynamic time warping algorithm. Extensive experiments are conducted on multiple benchmark datasets. The experimental results verify that the proposed method is robust to noise, articulated variations, and rigid transformations. Our method outperforms state-of-the-art methods in the comparisons of accuracy and efficiency.

This chapter is focused on some classical feature representations for image and video analysis. In particular, we will introduce the histogram-based features, texture features, and some local point features.

Object recognition, one of the important tasks of image recognition, mainly aimed at the recognition of visible images. It can accurately define and describe objects by attributes and features of geometric appearance, texture and material of images. In a broad sense, the recognition process can distinguish objects from backgrounds and other suspicious objects, such as cars and roads, that is the object detection.

In this chapter, we talk about one of the typical image understanding problems—cross-camera person re-identification. Some classical visual descriptors and metric learning algorithms for person re-identification are detailed.

In this chapter we firstly introduce the development and the main reasons of the success of deep learning, then the structure and principle of the deep CNN are explored, and several classical convolution network models are analyzed, finally two instances based on CNN architecture are given.

This chapter briefly introduces the background reading of scene classification and two topic models: LDA model and Topic Model using Belief Propagation (TMBP).

This chapter is devoted to some segmentation method of image and video. For image segmentation, five types of methods are detailed, including threshold segmentation, region-based segmentation, partial differential equation based segmentation, clustering based segmentation, and the graph theory based segmentation. For video segmentation, we shall introduce the motion region extraction method based on cumulative difference.