Search

Large-scale Text-to-Image (TTI) models have become a common approach for generating training data in various generative fields. However, visual hallucinations, which contain perceptually critical defects, remain a concern, especially in non-photorealistic styles like cartoon characters. We propose a novel visual hallucination detection system for cartoon character images generated by TTI models. Our approach leverages pose-aware in-context visual learning (PA-ICVL) with Vision-Language Models (VLMs), utilizing both RGB images and pose information. By incorporating pose guidance from a fine-tuned pose estimator, we enable VLMs to make more accurate decisions. Experimental results demonstrate significant improvements in identifying visual hallucinations compared to baseline methods relying solely on RGB images. This research advances TTI models by mitigating visual hallucinations, expanding their potential in non-photorealistic domains., Comment: 11 pages, 12 figures, 1 table, Project page: https://gh-bumsookim.github.io/Cartoon-Hallucinations-Detection/

In this paper, we first present the character texture generation system \textit{Minecraft-ify}, specified to Minecraft video game toward in-game application. Ours can generate face-focused image for texture mapping tailored to 3D virtual character having cube manifold. While existing projects or works only generate texture, proposed system can inverse the user-provided real image, or generate average/random appearance from learned distribution. Moreover, it can be manipulated with text-guidance using StyleGAN and StyleCLIP. These features provide a more extended user experience with enlarged freedom as a user-friendly AI-tool. Project page can be found at https://gh-bumsookim.github.io/Minecraft-ify/, Comment: 2 pages, 2 figures. Accepted as Spotlight to NeurIPS 2023 Workshop on Machine Learning for Creativity and Design

Face re-aging is a prominent field in computer vision and graphics, with significant applications in photorealistic domains such as movies, advertising, and live streaming. Recently, the need to apply face re-aging to non-photorealistic images, like comics, illustrations, and animations, has emerged as an extension in various entertainment sectors. However, the lack of a network that can seamlessly edit the apparent age in NPR images has limited these tasks to a naive, sequential approach. This often results in unpleasant artifacts and a loss of facial attributes due to domain discrepancies. In this paper, we introduce a novel one-stage method for face re-aging combined with portrait style transfer, executed in a single generative step. We leverage existing face re-aging and style transfer networks, both trained within the same PR domain. Our method uniquely fuses distinct latent vectors, each responsible for managing aging-related attributes and NPR appearance. By adopting an exemplar-based approach, our method offers greater flexibility compared to domain-level fine-tuning approaches, which typically require separate training or fine-tuning for each domain. This effectively addresses the limitation of requiring paired datasets for re-aging and domain-level, data-driven approaches for stylization. Our experiments show that our model can effortlessly generate re-aged images while simultaneously transferring the style of examples, maintaining both natural appearance and controllability., Comment: Accepted at CVPR 2024 AI4CC Workshop, Project Page: https://gh-bumsookim.github.io/ToonAging/

The pneumonia caused by Coronavirus disease (COVID-19) is one of major global threat and a number of detection and treatment procedures are suggested by the researchers for COVID-19. The proposed work aims to suggest an automated image processing scheme to extract the COVID-19 lesion from the lung CT scan images (CTI) recorded from the patients. This scheme implements the following procedures; (i) Image pre-processing to enhance the COVID-19 lesions, (ii) Image post-processing to extract the lesions, and (iii) Execution of a relative analysis between the extracted lesion segment and the Ground-Truth-Image (GTI). This work implements Firefly Algorithm and Shannon Entropy (FA+SE) based multi-threshold to enhance the pneumonia lesion and implements Markov-Random-Field (MRF) segmentation to extract the lesions with better accuracy. The proposed scheme is tested and validated using a class of COVID-19 CTI obtained from the existing image datasets and the experimental outcome is appraised to authenticate the clinical significance of the proposed scheme. The proposed work helped to attain a mean accuracy of >92% during COVID-19 lesion segmentation and in future, it can be used to examine the real clinical lung CTI of COVID-19 patients., Comment: 12 pages