Search

When there is a mismatch between the target identity and the driver identity, face reenactment suffers severe degradation in the quality of the result, especially in a few-shot setting. The identity preservation problem, where the model loses the detailed information of the target leading to a defective output, is the most common failure mode. The problem has several potential sources such as the identity of the driver leaking due to the identity mismatch, or dealing with unseen large poses. To overcome such problems, we introduce components that address the mentioned problem: image attention block, target feature alignment, and landmark transformer. Through attending and warping the relevant features, the proposed architecture, called MarioNETte, produces high-quality reenactments of unseen identities in a few-shot setting. In addition, the landmark transformer dramatically alleviates the identity preservation problem by isolating the expression geometry through landmark disentanglement. Comprehensive experiments are performed to verify that the proposed framework can generate highly realistic faces, outperforming all other baselines, even under a significant mismatch of facial characteristics between the target and the driver., In AAAI 2020

Background and objectives: Parkinson’s disease (PD) and schizophrenia often share symptomatology. Psychotic symptoms are prevalent in patients with PD, and similar motor symptoms with extrapyramidal signs are frequently observed in antipsychotic-naïve patients with schizophrenia as well as premorbid families. However, few studies have examined the relationship between PD and schizophrenia. We performed this study to evaluate whether genetic variants which increase PD risk influence the risk of developing schizophrenia, and vice versa. Materials and Methods: Two-sample Mendelian randomization (TSMR) with summary statistics from large-scale genome-wide association studies (GWAS) was applied. Summary statistics were extracted for these instruments from GWAS of PD and schizophrenia, Results: We found an increase in the risk of schizophrenia per one-standard deviation (SD) increase in the genetically-predicted PD risk (inverse-variance weighted method, odds ratio = 1.10, 95% confidence interval, 1.05−1.15, p = 3.49 × 10−5). The association was consistent in sensitivity analyses, including multiple TSMR methods, analysis after removing outlier variants with potential pleiotropic effects, and analysis after applying multiple GWAS subthresholds. No relationships were evident between PD and smoking or other psychiatric disorders, including attention deficit hyperactivity disorder, autism spectrum disorder, bipolar affective disorder, major depressive disorder, Alzheimer’s disease, or alcohol dependence. However, we did not find a reverse relationship, genetic variants increasing schizophrenia risk did not alter the risk of PD, Conclusions: Overall, our findings suggest that increased genetic risk of PD can be associated with increased risk of schizophrenia. This association supports the intrinsic nature of the psychotic symptom in PD rather than medication or environmental effects. Future studies for possible comorbidities and shared genetic structure between the two diseases are warranted.

The current evaluation protocol of long-tailed visual recognition trains the classification model on the long-tailed source label distribution and evaluates its performance on the uniform target label distribution. Such protocol has questionable practicality since the target may also be long-tailed. Therefore, we formulate long-tailed visual recognition as a label shift problem where the target and source label distributions are different. One of the significant hurdles in dealing with the label shift problem is the entanglement between the source label distribution and the model prediction. In this paper, we focus on disentangling the source label distribution from the model prediction. We first introduce a simple but overlooked baseline method that matches the target label distribution by post-processing the model prediction trained by the cross-entropy loss and the Softmax function. Although this method surpasses state-of-the-art methods on benchmark datasets, it can be further improved by directly disentangling the source label distribution from the model prediction in the training phase. Thus, we propose a novel method, LAbel distribution DisEntangling (LADE) loss based on the optimal bound of Donsker-Varadhan representation. LADE achieves state-of-the-art performance on benchmark datasets such as CIFAR-100-LT, Places-LT, ImageNet-LT, and iNaturalist 2018. Moreover, LADE outperforms existing methods on various shifted target label distributions, showing the general adaptability of our proposed method., Comment: CVPR 2021

Occupational attainment, which represents middle-age cognitive activities, is a known proxy marker of cognitive reserve for Alzheimer's disease. Previous genome-wide association studies have identified numerous genetic variants and revealed the genetic architecture of educational attainment, another marker of cognitive reserve. However, the genetic architecture and heritability for occupational attainment remain elusive. We performed a large-scale genome-wide association study of occupational attainment with 248 847 European individuals from the UK Biobank using the proportional odds logistic mixed model method. In this analysis, we defined occupational attainment using the classified job levels formulated in the UK Standard Occupational Classification system considering the individual professional skill and academic level. We identified 30 significant loci (P < 5 × 10−8); 12 were novel variants, not associated with other traits. Among them, four lead variants were associated with genes expressed in brain tissues by expression quantitative trait loci mapping from 10 brain regions: rs13002946, rs3741368, rs11654986 and rs1627527. The single nucleotide polymorphism-based heritability was estimated to be 8.5% (standard error of the mean = 0.004) and partitioned heritability was enriched in the CNS and brain tissues. Genetic correlation analysis showed shared genetic backgrounds between occupational attainment and multiple traits, including education, intelligence, leisure activities, life satisfaction and neuropsychiatric disorders. In two-sample Mendelian randomization analysis, we demonstrated that high occupation levels were associated with reduced risk for Alzheimer's disease [odds ratio (OR) = 0.78, 95% confidence interval (CI) = 0.65–0.92 in inverse variance weighted method; OR = 0.73, 95% CI = 0.57–0.92 in the weighted median method]. This causal relationship between occupational attainment and Alzheimer's disease was robust in additional sensitivity analysis that excluded potentially pleiotropic single nucleotide polymorphisms (OR = 0.72, 95% CI = 0.57–0.91 in the inverse variance weighted method; OR = 0.72, 95% CI = 0.53–0.97 in the weighted median method). Multivariable Mendelian randomization confirmed that occupational attainment had an independent effect on the risk for Alzheimer’s disease even after taking educational attainment into account (OR = 0.72, 95% CI = 0.54–0.95 in the inverse variance weighted method; OR = 0.68, 95% CI = 0.48–0.97 in the weighted median method). Overall, our analyses provide insights into the genetic architecture of occupational attainment and demonstrate that occupational attainment is a potential causal protective factor for Alzheimer's disease as a proxy marker of cognitive reserve.

In this study, an organic conjugated polymer-based photodetector was investigated as a candidate for the indirect-type radiation detector. In order to improve the photon to charge conversion efficiency, we selected the fullerene derivative indene-C60-bisadduct (ICBA) as an n-type semiconductor. The optimal process condition was investigated while varying the P3HT:ICBA blending ratio and spin-rate. The detector at the condition of P3HT:ICBA = 3:2 blending ratio and 900 rpm spin-rate showed the highest sensitivity of 1.35 mC/Gy $\cdot $ cm2 and the highest PCE of 2.93%. Compared to the common P3HT:PCBM detector, the best P3HT:ICBA detector showed 27.36% enhancement of sensitivity and 27.68% enhancement of the PCE.

Saliency Map, the gradient of the score function with respect to the input, is the most basic technique for interpreting deep neural network decisions. However, saliency maps are often visually noisy. Although several hypotheses were proposed to account for this phenomenon, there are few works that provide rigorous analyses of noisy saliency maps. In this paper, we firstly propose a new hypothesis that noise may occur in saliency maps when irrelevant features pass through ReLU activation functions. Then, we propose Rectified Gradient, a method that alleviates this problem through layer-wise thresholding during backpropagation. Experiments with neural networks trained on CIFAR-10 and ImageNet showed effectiveness of our method and its superiority to other attribution methods., Comment: Accepted at the 2019 ICCV Workshop on Interpreting and Explaining Visual AI Models (VXAI 2019)

Keyword spotting (KWS) plays a critical role in enabling speech-based user interactions on smart devices. Recent developments in the field of deep learning have led to wide adoption of convolutional neural networks (CNNs) in KWS systems due to their exceptional accuracy and robustness. The main challenge faced by KWS systems is the trade-off between high accuracy and low latency. Unfortunately, there has been little quantitative analysis of the actual latency of KWS models on mobile devices. This is especially concerning since conventional convolution-based KWS approaches are known to require a large number of operations to attain an adequate level of performance. In this paper, we propose a temporal convolution for real-time KWS on mobile devices. Unlike most of the 2D convolution-based KWS approaches that require a deep architecture to fully capture both low- and high-frequency domains, we exploit temporal convolutions with a compact ResNet architecture. In Google Speech Command Dataset, we achieve more than \textbf{385x} speedup on Google Pixel 1 and surpass the accuracy compared to the state-of-the-art model. In addition, we release the implementation of the proposed and the baseline models including an end-to-end pipeline for training models and evaluating them on mobile devices., In INTERSPEECH 2019

This paper focuses on high-frequency induction heating system for railway turnout. In the existing railway industry, most of the heating systems for railway turnout use the heating wire. It attached to the rail conducts the heat generated by the resistance directly to the rail. The heating wire, which has been applied for a long time, is advantageous in that it is simple in construction and easy to install. However, since it consumes a lot of power and has low heat conduction efficiency it requires a long operation time for perfect snow melting. Therefore, the propose of this study is the IH system as a new heating system to cope with existing heating system, and to improve electrical efficiency and heat efficiency through the IH system. When the IH system for turnout operates, the generated electric energy is converted into heat energy from the heating plate (SUS430), which is a component of the heating system. On the other hand, since the conversion from electric to thermal energy has a great influence on the heating, circuit analysis, electromagnetic field analysis and thermal analysis are required. Therefore, this paper has suggested a high-frequency induction heating system with high electrical and thermal efficiency to replace the conventional heating system.

We propose methods of addressing two critical issues of pedestrian detection: (i) occlusion of target objects as false negative failure, and (ii) confusion with hard negative examples like vertical structures as false positive failure. Our solutions to these two problems are general and flexible enough to be applicable to any single-stage detection models. We implement our methods into four state-of-the-art single-stage models, including SqueezeDet+ [22], YOLOv2 [17], SSD [12], and DSSD [8]. We empirically validate that our approach indeed improves the performance of those four models on Caltech pedestrian [4] and CityPersons dataset [25]. Moreover, in some heavy occlusion settings, our approach achieves the best reported performance. Specifically, our two solutions are as follows. For better occlusion handling, we update the output tensors of single-stage models so that they include the prediction of part confidence scores, from which we compute a final occlusion-aware detection score. For reducing confusion with hard negative examples, we introduce average grid classifiers as post-refinement classifiers, trainable in an end-to-end fashion with little memory and time overhead (e.g. increase of 1-5 MB in memory and 1-2 ms in inference time).

This paper presents a new document image dewarping method that removes geometric distortions in camera-captured document images. The proposed method does not directly use the text-line which has been the most widely used feature for the document dewarping. Instead, we use the discrete representation of text-lines and text-blocks which are the sets of connected components. Also, we model the geometric distortions caused by page curl and perspective view as the generalized cylindrical surfaces and camera rotation respectively. With these distortion models and the discrete representation of the features, we design a cost function whose minimization yields the parameters of the distortion model. In the cost function, we encode the properties of the pages such as text-block alignment, line-spacing, and the straightness of text-lines. By describing the text features using the sets of discrete points, the cost function can be easily defined and efficiently solved by Levenberg-Marquadt algorithm. Experiments show that the proposed method works well for the various layouts and curved surfaces, and compares favorably with the conventional methods on the standard dataset. HighlightsDocument dewarping is an important problem in camera-based OCR.We formulate the dewarping as an optimization problem.The proposed method performs dewarping in a fully automatic manner.The proposed method can handle various layouts of documents.Our method yields the improved OCR performances compared with other methods.

This paper deals with a real-time panorama algorithm for mobile camera systems. The proposed system generates the panoramic images and shows the intermediate results simultaneously on the mobile display. The proposed panorama system consists of feature point extraction, feature tracking, rotation matrix estimation, and image warping onto cylindrical surface. Feature points are extracted by fast Hessian detector in the low resolution and corner detector in the high resolution. Then, the feature points are tracked by matching between input frame and synthesized panorama image. The camera motion is modeled as a rotation matrix which is estimated using the tracked feature points. For realtime operation of panoramic image synthesis, we propose a method to estimate the rotation matrix using non-iterative least square method. We project the feature points of image onto the unit sphere surface in the world coordinates. This enables us to estimate the parameters of rotation matrix as the linear and non-iterative problem. Finally, we project the input frames onto panorama surface using the rotation matrix. We also implement a real-time display interface to show the intermediate panorama results while panorama image is being generated. Thus, the proposed panorama system paints the panorama canvas with real images. According to the experiments in the mobile systems such as mobile phone and tablet PC, the proposed system works well to generate the panoramic images in real-time.

Purpose The purpose of this study was to evaluate cerebral blood flow using arterial spin labeling (ASL) perfusion magnetic resonance (MR) imaging in patients with clinical seizure activity and determine its diagnostic performance in identifying seizure focus. Materials and methods Institutional Review Board of our hospital approved this retrospective study. Informed consent was waived. Clinical seizure focus was determined by a neurologist based on seizure semiology, electroencephalography, and conventional imaging modalities. The diagnostic performance of ASL perfusion MR imaging to identifying seizure focus compared to clinical seizure focus was analyzed. Results Clinical seizure focus was localized in 95% (42/44) of patients. The sensitivity and specificity of ASL perfusion MR imaging for identifying seizure focus were 74% (95% CI: 58%, 86%) (clinical seizure focus was localizable in 31 of 42 patients, including complete concordance in 10 patients and partial concordance in 21 patients) and 0% (95% CI: 0%, 84%) (for the two patients whose clinical seizure foci were not localizable, they were identified by ASL perfusion MR imaging), respectively. Thus, the overall accuracy of ASL perfusion MR imaging for localizing seizure focus was 70% (33/44). For 4 patients who had abnormal perfusion on ASL, their seizure foci based on ASL perfusion MR imaging were discordant with clinical seizure foci. Conclusion ASL perfusion MR imaging can provide information about perfusion status and important diagnostic clue in localizing seizure focus in patients with clinical seizure activity. It has the potential as a non-invasive complementary diagnostic tool for patients with clinical seizure activity.

This paper presents a hole-filling method based on spatial-temporal background information for view synthesis in 3D Video. The process of obtaining stationary background information is updated with a new background codebook in order to extract reliable temporal background modeling. An estimation of local background value is conducted to discriminate an adaptively boundary between the background region and the foreground region as well as to update the information about the hole region. The holes are filled by combining the spatial background information and the temporal background information. In addition, an exemplar-based inpainting method is used to fill the rest of holes, in which a priority function using background depth information is defined to determine the order in which the holes are filled. The experimental results demonstrated that the proposed algorithm synthesized satisfactory views regardless of video characteristics and type of hole region.

This paper presents video stabilization based on detection and adaptive alpha-trimmed means smoothing filtering of undesirable motion. The proposed algorithm consists of two stages: the detection of undesirable motion and smoothing filtering of detected undesired motion. To incorporate desired properties into the motion smoothing process, the local maximum and the local minimum are defined in a set composed of the parameters of accumulative global motion. Using the local information, the constraints on detecting undesirable motions are defined. Based on these constraints, the alpha parameter of the alpha-trimmed means filter is adjusted, so that the degree of motion smoothing in the reconstructed video sequence is controlled. The experimental results demonstrated the capability of the proposed algorithm.

In this paper, a novel hole filling approach bases on the edge orientation for new view synthesis is presented. The hole pixels after warping processing are generated from the background pixels guided by the direction of edges. To obtain the better visual quality of the generated image at the free view point, the priority of each pixels on hole's boundary is first calculated, then the depth pixels in a local rectangle centered at high-priority pixel are segmented by k-means prior to apply filling process. Thus, the natural textures of virtual view over the hole can be archived in less blur and distortion. The experimental results show that the proposed method provides better performance objectively as well as subjectively.

We propose a new image projection method in an attempt to reduce the perceptual distortion in panoramic image mosaics. Specifically, we reduce the stretching distortion of some image patches and bending of straight lines. Since the stretching distortion usually occurs when projecting a viewing sphere to the cylindrical image surface in an oblique direction, we propose to use an adjustable cylindrical surface to match the viewing direction with the equator of the cylindrical surface. Also, in order to find the trade-off between the stretching distortion and bending of straight lines, we also adjust the curvature of cylindrical surface according to the object of interest in the image. The warping function from the viewing sphere to the adjustable image surface is derived and the amount of distortion caused by this warping function is also defined. From the measure of distortion, the optimal pose of the cylindrical image plane and its curvature are determined, and the image on the viewing sphere is projected on the optimal plane. The experimental results show that the proposed method produces the panoramic image with less distortion than the existing methods.

In image stitching applications, it is very important to find a suitable warping function for the visual quality of a composite (stitching result). In this paper, we present a new mathematical criterion to select an optimal warping function among a set of possible candidates (e.g., parametric family). The proposed criterion can be considered as a direct view condition for image stitching , i.e., it is desirable that each part of the composite image looks like its corresponding input image. More specifically, we do not use an explicit modeling of a compositing surface, but, we focus on the differential properties of a warping function. That is we design a cost function so that the Jacobian matrix of a warping function is close to a shape preserving matrix such as rotation and reflection matrices. The proposed cost function can be effectively minimized by using Levenberg-Marquardt algorithm. The experimental results show that the proposed method results in visually pleasing stitched results because the original shape of each image is preserved in the composite.

Conventional image stitching methods were developed under the assumption or condition that (1) the optical center of a camera is fixed (fixed-optical-center case) or (2) the camera captures a plane target (plane-target case). Hence, users should know or test which condition is more appropriate for the given set of images and then select a right algorithm or try multiple stitching algorithms. We propose a unified framework for the image stitching and rectification problem, which can handle both cases in the same framework. To be precise, we model each camera pose with six parameters (three for the rotation and three for the translation) and develop a cost function that reflects the registration errors on a reference plane. The designed cost function is effectively minimized via the Levenberg–Marquardt algorithm. For the given set of images, when it is found that the relative camera motions between the images are large, the proposed method performs rectification of images and then composition using the rectified images; otherwise, the algorithm simply builds a visually pleasing result by selecting a viewpoint. Experimental results on synthetic and real images show that our method successfully performs stitching and metric rectification.