Search

Recently, diffusion models have made significant strides in synthesizing realistic 2D human images based on provided text prompts. Building upon this, researchers have extended 2D text-to-image diffusion models into the 3D domain for generating human textures (UV Maps). However, some important problems about UV Map Generative models are still not solved, i.e., how to generate personalized texture maps for any given face image, and how to define and evaluate the quality of these generated texture maps. To solve the above problems, we introduce a novel method, UVMap-ID, which is a controllable and personalized UV Map generative model. Unlike traditional large-scale training methods in 2D, we propose to fine-tune a pre-trained text-to-image diffusion model which is integrated with a face fusion module for achieving ID-driven customized generation. To support the finetuning strategy, we introduce a small-scale attribute-balanced training dataset, including high-quality textures with labeled text and Face ID. Additionally, we introduce some metrics to evaluate the multiple aspects of the textures. Finally, both quantitative and qualitative analyses demonstrate the effectiveness of our method in controllable and personalized UV Map generation. Code is publicly available via https://github.com/twowwj/UVMap-ID.

The research aims to investigate how the 2D slot game “Lucky Hero”, created for this research, could convey the story of Beowulf. The result of this study could provide experience and strategy for improving narrative design in casual games with similar mechanics. Also, this study is valuable to developers/designers who want to explore the potential of using casual games, especially slot games, as a storytelling tool. To answer the research question “How could the 2D slot game Lucky Hero convey the story of Beowulf”, a “research for design” qualitative study was conducted. The researchers examined theories related to narrative design, design in slot machine games, and folk mythology to develop the game. Playtests followed by interviews and adapted stimulation recall were used as data collection methods, and thematic analysis was used as the data analysis technique. The findings revealed that the existing design of "Lucky Hero" was not capable of effectively conveying the story of Beowulf to players. However, the study identified key elements that influenced players' understanding, such as graphics and game mechanics. The future narrative design of “Lucky Hero” should focus on improving immersion, character representation, and the connection between game mechanics and the story.

The research aims to investigate how the 2D slot game “Lucky Hero”, created for this research, could convey the story of Beowulf. The result of this study could provide experience and strategy for improving narrative design in casual games with similar mechanics. Also, this study is valuable to developers/designers who want to explore the potential of using casual games, especially slot games, as a storytelling tool. To answer the research question “How could the 2D slot game Lucky Hero convey the story of Beowulf”, a “research for design” qualitative study was conducted. The researchers examined theories related to narrative design, design in slot machine games, and folk mythology to develop the game. Playtests followed by interviews and adapted stimulation recall were used as data collection methods, and thematic analysis was used as the data analysis technique. The findings revealed that the existing design of "Lucky Hero" was not capable of effectively conveying the story of Beowulf to players. However, the study identified key elements that influenced players' understanding, such as graphics and game mechanics. The future narrative design of “Lucky Hero” should focus on improving immersion, character representation, and the connection between game mechanics and the story.

Generative Adversarial Networks (GANs), especially the recent style-based generators (StyleGANs), have versatile semantics in the structured latent space. Latent semantics discovery methods emerge to move around the latent code such that only one factor varies during the traversal. Recently, an unsupervised method proposed a promising direction to directly use the eigenvectors of the projection matrix that maps latent codes to features as the interpretable directions. However, one overlooked fact is that the projection matrix is non-orthogonal and the number of eigenvectors is too large. The non-orthogonality would entangle semantic attributes in the top few eigenvectors, and the large dimensionality might result in meaningless variations among the directions even if the matrix is orthogonal. To avoid these issues, we propose Householder Projector, a flexible and general low-rank orthogonal matrix representation based on Householder transformations, to parameterize the projection matrix. The orthogonality guarantees that the eigenvectors correspond to disentangled interpretable semantics, while the low-rank property encourages that each identified direction has meaningful variations. We integrate our projector into pre-trained StyleGAN2/StyleGAN3 and evaluate the models on several benchmarks. Within only $1\%$ of the original training steps for fine-tuning, our projector helps StyleGANs to discover more disentangled and precise semantic attributes without sacrificing image fidelity., Comment: ICCV23

Generative Neural Radiance Fields (GNeRF) based 3D-aware GANs have demonstrated remarkable capabilities in generating high-quality images while maintaining strong 3D consistency. Notably, significant advancements have been made in the domain of face generation. However, most existing models prioritize view consistency over disentanglement, resulting in limited semantic/attribute control during generation. To address this limitation, we propose a conditional GNeRF model incorporating specific attribute labels as input to enhance the controllability and disentanglement abilities of 3D-aware generative models. Our approach builds upon a pre-trained 3D-aware face model, and we introduce a Training as Init and Optimizing for Tuning (TRIOT) method to train a conditional normalized flow module to enable the facial attribute editing, then optimize the latent vector to improve attribute-editing precision further. Our extensive experiments demonstrate that our model produces high-quality edits with superior view consistency while preserving non-target regions. Code is available at https://github.com/zhangqianhui/TT-GNeRF., Comment: 13 pages

This paper proposes a gaze correction and animation method for high-resolution, unconstrained portrait images, which can be trained without the gaze angle and the head pose annotations. Common gaze-correction methods usually require annotating training data with precise gaze, and head pose information. Solving this problem using an unsupervised method remains an open problem, especially for high-resolution face images in the wild, which are not easy to annotate with gaze and head pose labels. To address this issue, we first create two new portrait datasets: CelebGaze and high-resolution CelebHQGaze. Second, we formulate the gaze correction task as an image inpainting problem, addressed using a Gaze Correction Module (GCM) and a Gaze Animation Module (GAM). Moreover, we propose an unsupervised training strategy, i.e., Synthesis-As-Training, to learn the correlation between the eye region features and the gaze angle. As a result, we can use the learned latent space for gaze animation with semantic interpolation in this space. Moreover, to alleviate both the memory and the computational costs in the training and the inference stage, we propose a Coarse-to-Fine Module (CFM) integrated with GCM and GAM. Extensive experiments validate the effectiveness of our method for both the gaze correction and the gaze animation tasks in both low and high-resolution face datasets in the wild and demonstrate the superiority of our method with respect to the state of the arts. Code is available at https://github.com/zhangqianhui/GazeAnimationV2, Comment: Accepted to TIP. arXiv admin note: text overlap with arXiv:2008.03834

Generative Neural Radiance Field (GNeRF) models, which extract implicit 3D representations from 2D images, have recently been shown to produce realistic images representing rigid/semi-rigid objects, such as human faces or cars. However, they usually struggle to generate high-quality images representing non-rigid objects, such as the human body, which is of a great interest for many computer graphics applications. This paper proposes a 3D-aware Semantic-Guided Generative Model (3D-SGAN) for human image synthesis, which combines a GNeRF with a texture generator. The former learns an implicit 3D representation of the human body and outputs a set of 2D semantic segmentation masks. The latter transforms these semantic masks into a real image, adding a realistic texture to the human appearance. Without requiring additional 3D information, our model can learn 3D human representations with a photo-realistic, controllable generation. Our experiments on the DeepFashion dataset show that 3D-SGAN significantly outperforms the most recent baselines. The code is available at https://github.com/zhangqianhui/3DSGAN, Comment: ECCV 2022. 29 pages

Controllable person image generation aims to produce realistic human images with desirable attributes such as a given pose, cloth textures, or hairstyles. However, the large spatial misalignment between source and target images makes the standard image-to-image translation architectures unsuitable for this task. Most state-of-the-art methods focus on alignment for global pose-transfer tasks. However, they fail to deal with region-specific texture-transfer tasks, especially for person images with complex textures. To solve this problem, we propose a novel Spatially-Adaptive Warped Normalization (SAWN) which integrates a learned flow-field to warp modulation parameters. It allows us to efficiently align person spatially-adaptive styles with pose features. Moreover, we propose a novel Self-Training Part Replacement (STPR) strategy to refine the model for the texture-transfer task, which improves the quality of the generated clothes and the preservation ability of non-target regions. Our experimental results on the widely used DeepFashion dataset demonstrate a significant improvement of the proposed method over the state-of-the-art methods on pose-transfer and texture-transfer tasks. The code is available at https://github.com/zhangqianhui/Sawn., Comment: 12 pages

Kai Wang,1,* Jinqiu Rao,1,* Tingting Zhang,1 Qing Gao,2 Jichao Zhang,3 Chenxi Guang,3 Liqin Ding,3 Feng Qiu1,3 1School of Chinese Materia Medica, Tianjin University of Traditional Chinese Medicine, Tianjin 301617, People’s Republic of China; 2School of Integrative Medicine, Tianjin University of Traditional Chinese Medicine, Tianjin 301617, People’s Republic of China; 3State Key Laboratory of Component-based Chinese Medicine,Tianjin University of Traditional Chinese Medicine, Tianjin, 301617, People’s Republic of China*These authors contributed equally to this workCorrespondence: Feng QiuSchool of Chinese Materia Medica, Tianjin University of Traditional Chinese Medicine, 10 Poyanghu Road, West Area, Tuanbo New Town, Jinghai District, Tianjin 301617, People’s Republic of ChinaTel/ Fax +86-022-59596223Email fengqiu20070118@163.comIntroduction: Berberrubine (BRB), an isoquinoline alkaloid, is a major constituent of medicinal plants Coptis chinensis Franch or Phellodendron chinense Schneid. BRB exhibits various pharmacological activities, whereas exposure to BRB may cause toxicity in experimental animals.Methods: In this study, we thoroughly investigated the liver injury induced by BRB in mice and rats. To explore the underlying mechanism, a study of the metabolic activation of BRB was conducted. Furthermore, covalent modifications of cysteine residues of proteins were observed in liver homogenate samples of animals after exposure to BRB, by application of an exhaustive proteolytic digestion method.Results: It was demonstrated that BRB-induced hepatotoxicities in a time- and dose-dependent manner, based on the biochemical parameters ALT and AST. H&E stained histopathological examination showed the occurrence of obvious edema in liver of mice after intraperitoneal (i.p.) administration of BRB at a single dose of 100 mg/kg. Slight hepatotoxicity was also observed in rats given the same doses of BRB after six weeks of gavage

In this paper we address the problem of unsupervised gaze correction in the wild, presenting a solution that works without the need for precise annotations of the gaze angle and the head pose. We have created a new dataset called CelebAGaze, which consists of two domains X, Y, where the eyes are either staring at the camera or somewhere else. Our method consists of three novel modules: the Gaze Correction module (GCM), the Gaze Animation module (GAM), and the Pretrained Autoencoder module (PAM). Specifically, GCM and GAM separately train a dual in-painting network using data from the domain $X$ for gaze correction and data from the domain $Y$ for gaze animation. Additionally, a Synthesis-As-Training method is proposed when training GAM to encourage the features encoded from the eye region to be correlated with the angle information, resulting in a gaze animation which can be achieved by interpolation in the latent space. To further preserve the identity information~(e.g., eye shape, iris color), we propose the PAM with an Autoencoder, which is based on Self-Supervised mirror learning where the bottleneck features are angle-invariant and which works as an extra input to the dual in-painting models. Extensive experiments validate the effectiveness of the proposed method for gaze correction and gaze animation in the wild and demonstrate the superiority of our approach in producing more compelling results than state-of-the-art baselines. Our code, the pretrained models and the supplementary material are available at: https://github.com/zhangqianhui/GazeAnimation., Comment: Accepted By ACMMM 2020

Facial attribute editing aims to manipulate attributes on the human face, e.g., adding a mustache or changing the hair color. Existing approaches suffer from a serious compromise between correct attribute generation and preservation of the other information such as identity and background, because they edit the attributes in the imprecise area. To resolve this dilemma, we propose a progressive attention GAN (PA-GAN) for facial attribute editing. In our approach, the editing is progressively conducted from high to low feature level while being constrained inside a proper attribute area by an attention mask at each level. This manner prevents undesired modifications to the irrelevant regions from the beginning, and then the network can focus more on correctly generating the attributes within a proper boundary at each level. As a result, our approach achieves correct attribute editing with irrelevant details much better preserved compared with the state-of-the-arts. Codes are released at https://github.com/LynnHo/PA-GAN-Tensorflow., Comment: Code: https://github.com/LynnHo/PA-GAN-Tensorflow

Gaze redirection aims at manipulating the gaze of a given face image with respect to a desired direction (i.e., a reference angle) and it can be applied to many real life scenarios, such as video-conferencing or taking group photos. However, previous work on this topic mainly suffers of two limitations: (1) Low-quality image generation and (2) Low redirection precision. In this paper, we propose to alleviate these problems by means of a novel gaze redirection framework which exploits both a numerical and a pictorial direction guidance, jointly with a coarse-to-fine learning strategy. Specifically, the coarse branch learns the spatial transformation which warps input image according to desired gaze. On the other hand, the fine-grained branch consists of a generator network with conditional residual image learning and a multi-task discriminator. This second branch reduces the gap between the previously warped image and the ground-truth image and recovers finer texture details. Moreover, we propose a numerical and pictorial guidance module~(NPG) which uses a pictorial gazemap description and numerical angles as an extra guide to further improve the precision of gaze redirection. Extensive experiments on a benchmark dataset show that the proposed method outperforms the state-of-the-art approaches in terms of both image quality and redirection precision. The code is available at https://github.com/jingjingchen777/CFGR, Comment: 12 pages, accepted by WACV 2021

Kai Wang,1,* Jinqiu Rao,1,* Tingting Zhang,1 Qing Gao,2 Jichao Zhang,3 Chenxi Guang,3 Liqin Ding,3 Feng Qiu1,3 1School of Chinese Materia Medica, Tianjin University of Traditional Chinese Medicine, Tianjin 301617, People’s Republic of China; 2School of Integrative Medicine, Tianjin University of Traditional Chinese Medicine, Tianjin 301617, People’s Republic of China; 3State Key Laboratory of Component-based Chinese Medicine,Tianjin University of Traditional Chinese Medicine, Tianjin, 301617, People’s Republic of China*These authors contributed equally to this workCorrespondence: Feng QiuSchool of Chinese Materia Medica, Tianjin University of Traditional Chinese Medicine, 10 Poyanghu Road, West Area, Tuanbo New Town, Jinghai District, Tianjin 301617, People’s Republic of ChinaTel/ Fax +86-022-59596223Email fengqiu20070118@163.comIntroduction: Berberrubine (BRB), an isoquinoline alkaloid, is a major constituent of medicinal plants Coptis chinensis Franch or Phellodendron chinense Schneid. BRB exhibits various pharmacological activities, whereas exposure to BRB may cause toxicity in experimental animals.Methods: In this study, we thoroughly investigated the liver injury induced by BRB in mice and rats. To explore the underlying mechanism, a study of the metabolic activation of BRB was conducted. Furthermore, covalent modifications of cysteine residues of proteins were observed in liver homogenate samples of animals after exposure to BRB, by application of an exhaustive proteolytic digestion method.Results: It was demonstrated that BRB-induced hepatotoxicities in a time- and dose-dependent manner, based on the biochemical parameters ALT and AST. H&E stained histopathological examination showed the occurrence of obvious edema in liver of mice after intraperitoneal (i.p.) administration of BRB at a single dose of 100 mg/kg. Slight hepatotoxicity was also observed in rats given the same doses of BRB after six weeks of gavage

Gaze correction aims to redirect the person's gaze into the camera by manipulating the eye region, and it can be considered as a specific image resynthesis problem. Gaze correction has a wide range of applications in real life, such as taking a picture with staring at the camera. In this paper, we propose a novel method that is based on the inpainting model to learn from the face image to fill in the missing eye regions with new contents representing corrected eye gaze. Moreover, our model does not require the training dataset labeled with the specific head pose and eye angle information, thus, the training data is easy to collect. To retain the identity information of the eye region in the original input, we propose a self-guided pretrained model to learn the angle-invariance feature. Experiments show our model achieves very compelling gaze-corrected results in the wild dataset which is collected from the website and will be introduced in details. Code is available at https://github.com/zhangqianhui/GazeCorrection.

Recent Image-to-Image Translation algorithms have achieved significant progress in neural style transfer and image attribute manipulation tasks. However, existing approaches require exhaustively labelling training data, which is labor demanding, difficult to scale up, and hard to migrate into new domains. To overcome such a key limitation, we propose Sparsely Grouped Generative Adversarial Networks (SG-GAN) as a novel approach that can translate images on sparsely grouped datasets where only a few samples for training are labelled. Using a novel one-input multi-output architecture, SG-GAN is well-suited for tackling sparsely grouped learning and multi-task learning. The proposed model can translate images among multiple groups using only a single commonly trained model. To experimentally validate advantages of the new model, we apply the proposed method to tackle a series of attribute manipulation tasks for facial images. Experimental results demonstrate that SG-GAN can generate image translation results of comparable quality with baselines methods on adequately labelled datasets and results of superior quality on sparsely grouped datasets. The official implementation is publicly available:https://github.com/zhangqianhui/Sparsely-Grouped-GAN., Comment: Accepted by ACMMM2018

Hengjun Lin, Xueke Qiu, Bo Zhang, Jichao Zhang Department of Tumor, Anus and Intestine, Jinhua People’s Hospital, Jinhua, Zhejiang 321000, China Background: Colorectal cancer is a malignant tumor with high death rate. Chemotherapy, radiotherapy and surgery are the three common treatments of colorectal cancer. For early colorectal cancer patients, postoperative adjuvant chemotherapy can reduce the risk of recurrence. For advanced colorectal cancer patients, palliative chemotherapy can significantly improve the life quality of patients and prolong survival. FOLFOX is one of the mainstream chemotherapies in colorectal cancer, however, its response rate is only about 50%. Methods: To systematically investigate why some of the colorectal cancer patients have response to FOLFOX therapy while others do not, we searched all publicly available database and combined three gene expression datasets of colorectal cancer patients with FOLFOX therapy. With advanced minimal redundancy maximal relevance and incremental feature selection method, we identified the biomarker genes. Results: A Support Vector Machine-based classifier was constructed to predict the response of colorectal cancer patients to FOLFOX therapy. Its accuracy, sensitivity and specificity were 0.854, 0.845 and 0.863, respectively. Conclusion: The biological analysis of representative biomarker genes suggested that apoptosis and inflammation signaling pathways were essential for the response of colorectal cancer patients to FOLFOX chemotherapy. Keywords: colorectal cancer, FOLFOX therapy, support vector machine, minimal redundancy maximal relevance, incremental feature selection, chemotherapy response

Cancer is a leading cause of disease and death around the world, and is thought to be the largest challenge to human beings in the 21st century due to the lack of efficient treatments. A promising strategy for discovering anti-cancer drug candidates is to develop novel sialyltransferase transition-state analogue inhibitors as potential anti-metastasis drugs, to control cancer cell metastasis, and to prevent primary tumours turning into deadly secondary tumours. This thesis describes work towards the synthesis of sialyltransferase transition-state analogue inhibitors 15a-i, and in particular, three key fragments: α-hydroxyphosphonate derivatives, the protected cytidine component and the key phosphoryl linkage. The α-hydroxyphosphonate derivatives were prepared as both key components of the sialyltransferase transition-state analogue inhibitors and also as bioactive molecules themselves. Nine diethyl α-hydroxyphosphonate derivatives 16a-i, nine dibenzyl α- hydroxyphosphonate derivatives 17a-i and one hydroxyphosphonic acid (29), were prepared in an efficient, inexpensive and readily accessible manner. Six of the compounds were novel, while another two were novel at the commencement of the project, but were described in the literature during the course of the project. Biological evaluation of the α-hydroxyphosphonate derivatives showed that selected examples exhibited promising protein kinase A inhibitory activity of 85-100% when tested at 100 μg/ml. A large scale (overall more than 10 g) of the tri-N/O-acetylated cytidine (18) was obtained as the key precursor for the phosphoramidite coupling step in a highest overall yield of 64%. The two intermediate products, the silyl protected cytidine (31) and silyl protected tri-N/O-acetylated cytidine (32), were fully characterised and their reaction conditions optimised to give excellent yields of product (>90%). Moreover, the successful synthesis of the 2’, 3’-O-isopropylidene-cytidine (33) in 78% yield provided a simplified

2-Cyanoethyl N,N,N′,N′-tetraisopropylphosphorodiamidite is a colorless viscous liquid, which is soluble in most organic solvents. It is a widely used phosphitylating reagent for the preparation of various phosphorylated biomolecules, such as nucleoside carbohydrate conjugates, phospholipids and glycopeptides. In particular, this reagent is highly effective for automated solid-phase DNA/RNA oligonucleotide synthesis. 2-Cyanoethyl N,N,N′,N′-tetraisopropylphosphorodiamidite has shown great utility in the coupling of nucleobases or carbohydrates via their phosphotriesters in the presence of activators such as 1H-tetrazole, in moderate yields under mild conditions. Additionally, 2-cyanoethyl N,N,N′,N′-tetraisopropylphosphorodiamidite is cheaper and more stable than 2- cyanoethyl N,N-diisopropylchlorophosphorodiamidite, the other commonly used phosphinylating reagent. 2-Cyanoethyl N,N,N′,N′-tetraisopropylphosphorodiamidite is commercially available but can also be prepared in an inexpensive manner using a two-step, one-pot procedure and purified by vacuum distillation (Scheme [1]).

2-Cyanoethyl N,N,N′,N′-tetraisopropylphosphorodiamidite is a colorless viscous liquid, which is soluble in most organic solvents. It is a widely used phosphitylating reagent for the preparation of various phosphorylated biomolecules, such as nucleoside carbohydrate conjugates, phospholipids and glycopeptides. In particular, this reagent is highly effective for automated solid-phase DNA/RNA oligonucleotide synthesis. 2-Cyanoethyl N,N,N′,N′-tetraisopropylphosphorodiamidite has shown great utility in the coupling of nucleobases or carbohydrates via their phosphotriesters in the presence of activators such as 1H-tetrazole, in moderate yields under mild conditions. Additionally, 2-cyanoethyl N,N,N′,N′-tetraisopropylphosphorodiamidite is cheaper and more stable than 2- cyanoethyl N,N-diisopropylchlorophosphorodiamidite, the other commonly used phosphinylating reagent. 2-Cyanoethyl N,N,N′,N′-tetraisopropylphosphorodiamidite is commercially available but can also be prepared in an inexpensive manner using a two-step, one-pot procedure and purified by vacuum distillation (Scheme [1]).