Search

Your search keyword '"LIU, ZIWEI"' showing total 131 results
Start Over Author "LIU, ZIWEI" Publisher ieee
131 results on '"LIU, ZIWEI"'

8. Panoptic Video Scene Graph Generation

Author

Yang, Jingkang, primary, Peng, Wenxuan, additional, Li, Xiangtai, additional, Guo, Zujin, additional, Chen, Liangyu, additional, Li, Bo, additional, Ma, Zheng, additional, Zhou, Kaiyang, additional, Zhang, Wayne, additional, Loy, Chen Change, additional, and Liu, Ziwei, additional

Published
2023
Full Text
View/download PDF

25. Versatile Multi-Modal Pre-Training for Human-Centric Perception

Author

Hong, Fangzhou, Pan, Liang, Cai, Zhongang, and Liu, Ziwei

Subjects
FOS: Computer and information sciences, Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition
Abstract

Human-centric perception plays a vital role in vision and graphics. But their data annotations are prohibitively expensive. Therefore, it is desirable to have a versatile pre-train model that serves as a foundation for data-efficient downstream tasks transfer. To this end, we propose the Human-Centric Multi-Modal Contrastive Learning framework HCMoCo that leverages the multi-modal nature of human data (e.g. RGB, depth, 2D keypoints) for effective representation learning. The objective comes with two main challenges: dense pre-train for multi-modality data, efficient usage of sparse human priors. To tackle the challenges, we design the novel Dense Intra-sample Contrastive Learning and Sparse Structure-aware Contrastive Learning targets by hierarchically learning a modal-invariant latent space featured with continuous and ordinal feature distribution and structure-aware semantic consistency. HCMoCo provides pre-train for different modalities by combining heterogeneous datasets, which allows efficient usage of existing task-specific human data. Extensive experiments on four downstream tasks of different modalities demonstrate the effectiveness of HCMoCo, especially under data-efficient settings (7.16% and 12% improvement on DensePose Estimation and Human Parsing). Moreover, we demonstrate the versatility of HCMoCo by exploring cross-modality supervision and missing-modality inference, validating its strong ability in cross-modal association and reasoning., Comment: CVPR 2022; Project Page https://hongfz16.github.io/projects/HCMoCo.html; Codes available at https://github.com/hongfz16/HCMoCo

Published
2022
Full Text
View/download PDF

26. Conditional Prompt Learning for Vision-Language Models

Author

Zhou, Kaiyang, Yang, Jingkang, Loy, Chen Change, and Liu, Ziwei

Subjects
FOS: Computer and information sciences, Computer Science - Machine Learning, Artificial Intelligence (cs.AI), Computer Science - Computation and Language, Computer Science - Artificial Intelligence, Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition, Computation and Language (cs.CL), Machine Learning (cs.LG)
Abstract

With the rise of powerful pre-trained vision-language models like CLIP, it becomes essential to investigate ways to adapt these models to downstream datasets. A recently proposed method named Context Optimization (CoOp) introduces the concept of prompt learning -- a recent trend in NLP -- to the vision domain for adapting pre-trained vision-language models. Specifically, CoOp turns context words in a prompt into a set of learnable vectors and, with only a few labeled images for learning, can achieve huge improvements over intensively-tuned manual prompts. In our study we identify a critical problem of CoOp: the learned context is not generalizable to wider unseen classes within the same dataset, suggesting that CoOp overfits base classes observed during training. To address the problem, we propose Conditional Context Optimization (CoCoOp), which extends CoOp by further learning a lightweight neural network to generate for each image an input-conditional token (vector). Compared to CoOp's static prompts, our dynamic prompts adapt to each instance and are thus less sensitive to class shift. Extensive experiments show that CoCoOp generalizes much better than CoOp to unseen classes, even showing promising transferability beyond a single dataset; and yields stronger domain generalization performance as well. Code is available at https://github.com/KaiyangZhou/CoOp., CVPR 2022. Update: Adds results on the DOSCO (DOmain Shift in COntext) benchmark

Published
2022
Full Text
View/download PDF

42. Energy-Based Open-World Uncertainty Modeling for Confidence Calibration

Author

Wang, Yezhen, Li, Bo, Che, Tong, Zhou, Kaiyang, Liu, Ziwei, and Li, Dongsheng

Subjects
FOS: Computer and information sciences, Computer Science - Machine Learning, Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition, Machine Learning (cs.LG)
Abstract

Confidence calibration is of great importance to the reliability of decisions made by machine learning systems. However, discriminative classifiers based on deep neural networks are often criticized for producing overconfident predictions that fail to reflect the true correctness likelihood of classification accuracy. We argue that such an inability to model uncertainty is mainly caused by the closed-world nature in softmax: a model trained by the cross-entropy loss will be forced to classify input into one of $K$ pre-defined categories with high probability. To address this problem, we for the first time propose a novel $K$+1-way softmax formulation, which incorporates the modeling of open-world uncertainty as the extra dimension. To unify the learning of the original $K$-way classification task and the extra dimension that models uncertainty, we propose a novel energy-based objective function, and moreover, theoretically prove that optimizing such an objective essentially forces the extra dimension to capture the marginal data distribution. Extensive experiments show that our approach, Energy-based Open-World Softmax (EOW-Softmax), is superior to existing state-of-the-art methods in improving confidence calibration., Comment: ICCV 2021 (Poster)

Published
2021
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results