Search

Your search keyword '"Jin, Xiaoyu"' showing total 527 results
Start Over Author "Jin, Xiaoyu" Publication Year Range Last 50 years
527 results on '"Jin, Xiaoyu"'

1. Alignment is All You Need: A Training-free Augmentation Strategy for Pose-guided Video Generation

Author

Jin, Xiaoyu, Xu, Zunnan, Ou, Mingwen, and Yang, Wenming

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

Character animation is a transformative field in computer graphics and vision, enabling dynamic and realistic video animations from static images. Despite advancements, maintaining appearance consistency in animations remains a challenge. Our approach addresses this by introducing a training-free framework that ensures the generated video sequence preserves the reference image's subtleties, such as physique and proportions, through a dual alignment strategy. We decouple skeletal and motion priors from pose information, enabling precise control over animation generation. Our method also improves pixel-level alignment for conditional control from the reference character, enhancing the temporal consistency and visual cohesion of animations. Our method significantly enhances the quality of video generation without the need for large datasets or expensive computational resources., Comment: CVG@ICML 2024

Published
2024

2. VmambaIR: Visual State Space Model for Image Restoration

Author

Shi, Yuan, Xia, Bin, Jin, Xiaoyu, Wang, Xing, Zhao, Tianyu, Xia, Xin, Xiao, Xuefeng, and Yang, Wenming

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

Image restoration is a critical task in low-level computer vision, aiming to restore high-quality images from degraded inputs. Various models, such as convolutional neural networks (CNNs), generative adversarial networks (GANs), transformers, and diffusion models (DMs), have been employed to address this problem with significant impact. However, CNNs have limitations in capturing long-range dependencies. DMs require large prior models and computationally intensive denoising steps. Transformers have powerful modeling capabilities but face challenges due to quadratic complexity with input image size. To address these challenges, we propose VmambaIR, which introduces State Space Models (SSMs) with linear complexity into comprehensive image restoration tasks. We utilize a Unet architecture to stack our proposed Omni Selective Scan (OSS) blocks, consisting of an OSS module and an Efficient Feed-Forward Network (EFFN). Our proposed omni selective scan mechanism overcomes the unidirectional modeling limitation of SSMs by efficiently modeling image information flows in all six directions. Furthermore, we conducted a comprehensive evaluation of our VmambaIR across multiple image restoration tasks, including image deraining, single image super-resolution, and real-world image super-resolution. Extensive experimental results demonstrate that our proposed VmambaIR achieves state-of-the-art (SOTA) performance with much fewer computational resources and parameters. Our research highlights the potential of state space models as promising alternatives to the transformer and CNN architectures in serving as foundational frameworks for next-generation low-level visual tasks., Comment: 23 pages

Published
2024

3. LLMRA: Multi-modal Large Language Model based Restoration Assistant

Author

Jin, Xiaoyu, Shi, Yuan, Xia, Bin, and Yang, Wenming

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

Multi-modal Large Language Models (MLLMs) have a significant impact on various tasks, due to their extensive knowledge and powerful perception and generation capabilities. However, it still remains an open research problem on applying MLLMs to low-level vision tasks. In this paper, we present a simple MLLM-based Image Restoration framework to address this gap, namely Multi-modal Large Language Model based Restoration Assistant (LLMRA). We exploit the impressive capabilities of MLLMs to obtain the degradation information for universal image restoration. By employing a pretrained multi-modal large language model and a vision language model, we generate text descriptions and encode them as context embedding with degradation information for the degraded image. Through the proposed Context Enhance Module (CEM) and Degradation Context based Transformer Network (DC-former), we integrate these context embedding into the restoration network, contributing to more accurate and adjustable image restoration. Based on the dialogue with the users, our method leverages image degradation priors from MLLMs, providing low-level attributes descriptions of the input low-quality images and the restored high-quality images simultaneously. Extensive experiments demonstrate the superior performance of our LLMRA in universal image restoration tasks.

Published
2024

4. The Building Data Genome Directory -- An open, comprehensive data sharing platform for building performance research

Author

Jin, Xiaoyu, Fu, Chun, Kazmi, Hussain, Balint, Atilla, Canaydin, Ada, Quintana, Matias, Biljecki, Filip, Xiao, Fu, and Miller, Clayton

Subjects
Statistics - Applications
Abstract

The building sector plays a crucial role in the worldwide decarbonization effort, accounting for significant portions of energy consumption and environmental effects. However, the scarcity of open data sources is a continuous challenge for built environment researchers and practitioners. Although several efforts have been made to consolidate existing open datasets, no database currently offers a comprehensive collection of building data types with all subcategories and time granularities (e.g., year, month, and sub-hour). This paper presents the Building Data Genome Directory, an open data-sharing platform serving as a one-stop shop for the data necessary for vital categories of building energy research. The data directory is an online portal (http://buildingdatadirectory.org/) that allows filtering and discovering valuable datasets. The directory covers meter, building-level, and aggregated community-level data at the spatial scale and year-to-minute level at the temporal scale. The datasets were consolidated from a comprehensive exploration of sources, including governments, research institutes, and online energy dashboards. The results of this effort include the aggregation of 60 datasets pertaining to building energy ontologies, building energy models, building energy and water data, electric vehicle data, weather data, building information data, text-mining-based research data, image data of buildings, fault detection diagnosis data and occupant data. A crowdsourcing mechanism in the platform allows users to submit datasets they suggest for inclusion by filling out an online form. This directory can fuel research and applications on building energy efficiency, which is an essential step toward addressing the world's energy and environmental challenges.

Published
2023
Full Text
View/download PDF

24. Effects of affective factors on cognitive control engagement in childhood and adulthood

Author

Jin, Xiaoyu, Chevalier, Nicolas, and Auyeung, Bonnie

Subjects
children's attentional resources control, motivational contexts, emotional contexts, attentional resources, children's attentional management, social rewards, non-social rewards, reward expectation
Abstract

Childhood is a period of protracted gains in cognitive control, with increasingly flexible and efficient engagement of cognitive resources as a function of task demands. In particular, children progressively shift from relying on "as-needed" reactive control engagement at preschool age to more anticipatory proactive control engagement by middle childhood. Cognitive control is usually implemented in emotionally or motivationally charged contexts in real-world, which may present both challenges and opportunities for control engagement during childhood. The aim of this thesis was to examine how affective factors influence cognitive control in children, particularly the temporal dynamics of control engagement. Using electroencephalography (EEG), Study 1 showed that both positive stimuli and external rewards encouraged children and adults to engage practive control in a cued task-switching paradigm, but in different ways: provision of external rewards promoted proactive cue alertness, whereas the presence of positive stimuli enhanced proactive task selection. Study 2 leveraged fMRI to examine the brain circuitry supporting these beneficial effects in young adults. Provising of external rewards elicited greater cue-locked activation in the lateral frontal cortex, while no effect of positive stimuli on brain activation was detected. Study 3 used pupillometry to further investigate the effects of different reward types on control engagement. Both performance contingent and non-contingent rewards, provided as either a sustained incentive context or a transient incentive, improved proactive control engagement in the AX-CPT in childhood as well as adulthood. Willingness to engage cognitive effort, which was not influenced by reward contingency, mediated partially the effect of age on proactive control engagement. Finally, Study 4 compared the effects of social and non-social rewards on cognitive control engagement in an online version of AX-CPT, with other important reward dimensions being balanced between reward types. Social rewards enhanced cognitive control performance but did not influence whether control was engaged reactively or proactively in both preschool and school-aged children, whereas no effect of non-social rewards was observed. Together, these findings indicate that external rewards and, to a lesser extent, positive stimuli, can help children strategically upregulate cognitive control engagement by promoting a proactive mode of control, although these effects do not always translate into behavioural benefits. They have important implications for assessing and promoting cognitive control both in and outside of the lab.

Published
2021
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results